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Abstract 

We study the following doubly critical Schrodinger system 




JV \{0}, 

where N > 3, Ai,A 2 <= (0, {N ~ 2)2 ), 2* = ^ and a > l,/3 > 1 satisfying 
a + /3 — 2*. This problem is related to coupled nonlinear Schrodinger equations 
with critical exponent for Bose-Einstein condensate. For different ranges of 
N, a, /3 and v > 0, we obtain positive ground state solutions via some quite 
different methods, which are all radially symmetric. It turns out that the least 
energy level depends heavily on the relations among a, (3 and 2. Besides, for 
sufficiently small v > 0, positive solutions are also obtained via a variational 
perturbation approach. Note that the Palais-Smale condition can not hold 
for any positive energy level, which makes the study via variational methods 
rather complicated. 
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1 Introduction 



In this paper we consider solitary wave solutions of coupled nonlinear Schrodinger 
equations, known in the literature as Gross-Pitaevskii equations ( |19[ 134] ): 

= A$i - a(x)$! + /ii|$i| 2 $i + v|* 2 | 2 *i, x G R w , * > 0, 
( -if $ 2 = A$ 2 - 6(x)$ 2 + /i 2 |$ 2 | 2 $2 + ^|$i| 2 $2, x G R N , i > 0, 
' <l>, <!•;:.-•./:. .I.. J -1,2, 
k <&j(a:,t) -> 0, as \x\ -> +oo, t > 0, j = 1,2, 

where i is the imaginary unit, a(x), b(x) are potential functions, p\, /12 > and 
f 7^ is a coupling constant. System (jl.ll) appears in many physical problems, 
especially in nonlinear optics. Physically, the solution <£>j denotes the j th com- 
ponent of the beam in Kerr- like photorefractive media (see [3]). The positive 
constant /ij is for self-focusing in the j th component of the beam. The coupling 
constant v is the interaction between the two components of the beam. Prob- 
lem (|1.1[) also arises in the Hartree-Fock theory for a double condensate, i.e., 
a binary mixture of Bose-Einstein condensates in two different hyperfine states 
|1) and |2) (see [2]). Physically, are the corresponding condensate ampli- 
tudes, and v are the intraspecies and interspecies scattering lengths. The 
sign of v determines whether the interactions of states |1) and |2) are repulsive 
or attractive, i.e., the interaction is attractive if v > 0, and the interaction is 
repulsive if v < 0, where the two states are in strong competition. 

To obtain solitary wave solutions of system (|l.lj) . we set $i (x, t) — e zXlt u(x) 
and $ 2 (x,t) = e lX2t v{x). Write Vi(x) = a(x) + Ai and V 2 {x) = b(x) + A 2 for 
convenience, and we are only interested in nonnegative solutions, then system 
(jl.ip is reduced to the following elliptic system 

-Au + Vi(x)u = niu 3 + vuv 2 , xeR N , 

-Av + V 2 (x)v = fi 2 v 3 + wu 2 , iGl", (1.2) 
u > 0, v > in R N , u(x),v(x) -4 as |i| -t oo. 

This Bose-Einstein condensate type system (|1.2p is a special case of the following 
problem 

-Am + Vi(x)u = mu 2 ^ 1 + vu p - l v p , x e R N , 
-Av + V 2 (x)v = fJ , 2 v 2p - 1 + vv p - 1 uP : x&R N , (1.3) 
u > 0,« > in R N , u(x),v(x) — > as \x\ — > oo, 

where p > 1 and p<2*/2if7V>3,2* = is the critical Sobolev exponent. 
If p = 2, then (|1.3[) turns to be the cubic system (|1.2I) . For further introduction 
about this problem, readers can also see the survey articles [TH1 HO] > which also 
contain information about the physical relevance of non-cubic nonlinearitics 
(e.g. quintic). For the subcritical case p < 2*/2, the existence and multiplicity 
of solutions to (| 1 . 3|) have been widely studied under different assumptions on Vi 
and j/, see E2 [23 [20 [22 EH |34] and references therein. 
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Remark that, all the papers mentioned above deal with the subcritical case. 
To the best of our knowledge, there is no existence results for (|1.3[) in the critical 
case 2p = 2* in the literature. 

In this paper, we study (ll.3[) in critical case where N > 3 and 2p = 2*. 
In this case, if Vi(x) = A; are nonzero constants with the same sign, then by 
Pohozaev identity, we easily conclude that any solution (u, v) of (|1.3[) satisfy 
Jjjjv Xiu 2 + X 2 v 2 — 0, and so (u, v) = (0, 0). Hence we do not consider the case 
Vi(x) = \i here, and in the sequel we assume that Vi{x) = — J^f^" are Hardy type 
potentials. The Hardy type potentials, which arise in several physical contexts 
(e.g., in nonrelativistic quantum mechanics, molecular physics, quantum cos- 
mology, linearization of combustion models), do not belong to the kato's class, 
so they cannot be regarded as a lower order perturbation term. In particular, 
any nontrivial solutions of (jl.3p with Vi(x) — — y§fr are singular at x = 0. For 
the sake of simplicity, in the sequel we assume that (i± = fJ.2 = 1- Then, to 
study (|1.3[) with 2p — 2*, Vi(x) = — and \i\ = /12 — 1, we turn to study the 
following general problem 

-Am- Msu = u 2 *- 1 + vau^v 13 , x e R N , 
~Av- ^jv = v 2 '- 1 +vpu a v fi - 1 , iGl 1 , (1.4) 
u, v e D 1 ' 2 (R N ), u,v>0 in R N \ {0}, 

where N> 3, Ai,A 2 G (0,A N ), A N := ^^i!, 

a>l, p>l, a + p = 2\ (1.5) 
and D 1 ' 2 (M Ar ) is the completion of Cg°(R N ) with respect to the norm 




Note that if a = /? = p = 2*/2, then (TOl) turns to be ([T5]) with Vi(x) = —£h 
and Hi = jU2 = 1- The mathematical interest in system (|1.4I) relies on their 
double criticality, due to the fact that both the exponent of the nonlinearitics 
(which is critical in the sense of the Sobolev embedding) and the singularities 
share the same order of homogeneity as the Laplacian. The main goal of this 
paper is to study the existence and radial symmetry of ground state solutions of 
system , where the ground state solution is defined in Definition [TTT] below. 
Recall that Ai, A2 G (0, Ajy), from Hardy inequality 

Ajv / ^alx< [ \Vu\ 2 dx, Vu G D 1 ' 2 (K JV ), (1.6) 

JR« \X\ Jtrn 

we see that || • i — 1,2, are equivalent norms to || • ||, where 

Nil- / \Vu\ 2 ~^u 2 dx. (1.7) 
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Denote the norm of L P (R N ) by \u\ p = (J„ N \u\ p dx)p . The case of a single 
equation has been deeply investigated in the literature. In particular, by [33] , 
the problem 

\u{x) e D^ 2 (R N ), u > in R N \ {0}, 
has exactly an one-dimensional C 2 manifold of positive solutions given by 

Z l = \zUx)= f i- E ^zl(-), fi>o\, (1.9) 



where 



,;u> = ~ (i.i.D 

4o A . 

1 + |d 2 ~^ 



a Ai = ^ - - A * m d A*) = . Moreover, all posi- 



tive solutions of (|1.8j) satisfy 



* (A ' } = „ gJ J&,\ { o } W = 1# = " (^VJ * (L11) 
and 

where 5 is the sharp constant of L> 1 ' 2 (R Ar ) <^-> i 2 *(R A ') 

/ \Vu\ 2 dx>S\ ( \uf dx) , (1.13) 



N -\ 

4A, 



and 



|ll«llL-|r / "I"' </•'■■ ' = 1-^ (1-14.) 



see [33] . There are also many papers working on related equations with a Hardy 
type potential and a critical nonlinearity, we refer readers to [2J IJH1 [21] and 
references therein. 

We call a solution (u, v) of (|1.4[) nontrivial if both u ^ and w ^ 0; we call 
a solution (u, w) positive if both u > and v > in R^ \ {0}; we call a solution 
(u, v) semi-trivial if (u,v) is a type of (u, 0) or (0, w). 

One of the difficulties in the study of (]1.4[) is that it has semi-trivial solutions 
(z*, 0) and (0, z 2 ). Here, we are only interested in nontrivial solutions of (jl.4p . 
Define B := D^ 2 (R N ) x D 1 ^ 2 (R JV ) with norm 

\\(u,v)\g :=HL + IHIa 2 - 
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Then nontrivial solutions of (|1.4p can be found as nontrivial critical points of 
the C 1 functional J„ : D — > K, where 

J v (u,«):=/ Al («)+JA a (»)-i/ / \u\ a \vfdx. 

JR N 

Another difhculty is the failure of the Palais-Smale condition, which makes 
the study of (|1.4p very tough. Since (|1.4j) is invariant under the transforma- 

...... . N — 2 . . Y-2 . 

tion (u(x),v(x)) n- (/i^ 2- u[fj,x), v(fj,x)), where fi > 0, it is easy to see 
that the Palais-Smale condition ((PS) condition for short) can not hold for 
any energy level c > 0. In fact, assume by contradiction that (PS) condi- 
tion holds for some c > 0, and let (u n ,v n ) be a (PS) C sequence, that is, 
J v (u n ,v n ) — » c and J' v (u n ,v n ) — > as n — > oo. Then up to a subsequence, 
we may assume that (u n ,v n ) — > (u,v) strongly in ED. Define (u n (x),v n (x)) :— 

2 iV 2 

(n — u„(nx), n - 2~~ v n (nx)), then it is easy to check that (u n , v n ) is also a (PS) C 
sequence and (u n ,v n ) (0,0) weakly in D. Since (PS) C condition holds, we 
have (u n , v n ) — > (0, 0) strongly in ED, which contradicts with c > 0. 

Definition 1.1. We say a solution (uo,Vo) of is a ground state solution if 
(uq,vq) is nontrivial and J u (uq,v$) < J v (u, v) for any other nontrivial solution 
(u,v)of(n$. 



To obtain ground state solutions of (|1.4p . as in [22], we define 

My := \(u,v) G B : u ^ 0,w ^ 0, ||tt|| A = / (\uf + va\u\ a \vf), 

(\v\ 2 ' +vp\u\ a \v\P)}. (1.15) 



Then any nontrivial solutions of (|1.4|) has to belong to AC. Take ip, ip £ Co°(M ) 
with ip,ip^0 and supp((p) H supp(ip) = 0, then there exist ti,£2 > such that 
i 2 V0 ^ ^ for any v ^ 0. So AC 7^ 0. We set 

C := inf J„( U , v) = inf 1 (||u||^ + |M| 2 A2 ) . (1.16) 
(u,u)eAf„ (ti,u)&A/L, 

By p. lip we have 

||«||a, > 5(Ai)|u||. Vue D h2 (R N ), i = 1,2. (1.17) 

Then it is easy to see that c„ > for all v. Moreover, if (uq,Uo) is a nontrivial 
solution satisfying J„(uo,Vq) = c v , then (uq,vq) is a ground state solution. 
Our first result is concerned with ground state solutions with energy below 
^min{S(X 1 ) N /^S(X 2 m. 

Theorem 1.1. Assume that N > 3, Ai, A2 € (0,Ajv) M.5\) hold. 
(1) If v < 0, £/ien cv = -^■S'(Ai) Ar / 2 + jjS(\2) N / 2 , and c v can not be attained. 



5 



(2) Let 



1 



1 + max 



Ajv — A2 Ajy — Ai 



> 0, (1.18) 



then for all v > i/q, has a positive ground state solution (u v , v v ) fD, 

which is radially symmetric and satisfies 

J v {u u ,v v ) = c v < ^mhx{s(X 1 ) N / 2 , 5(A 2 ) Ar / 2 }. (1.19) 

(3) If one of the following conditions 

(Ci) N > 5 and max{a,@} < 2, 
(C 2 ) Ai < A 2 and a < 2, 
(C 3 ) A 2 < Ai and [3 < 2, 

holds, then for all v > 0, has a positive ground state solution (u„, v v ) € 

B, which is radially symmetric and satisfies H1.19\) . 

Now we want to obtain ground state solutions with energy above the value 
max{S(\i) N / 2 , S(\2) N / 2 }, which seems much more interesting to us. To this 
goal, by Theorem ll.il (2)-(3) we have to assume that min{a, /?} > 2 and v > 
is small. In this case, since 4 < a + /3 = 2*, soA r = 3or7V = 4. Moreover, if 
N = 4, then we must have a = j3 = 2. Note that if N = 4 and a = j3 = 2, then 
(|1.4p turns to be the following cubic system 

-Au - j^u = u 3 + 2vuv 2 , x e M 4 , 

-Av- M^v = v 3 + 2vu 2 v, i£l 4 , (1.20) 
^.ueD^I 4 ), m, u > in M 4 \ {0}, 

which is just the Bose-Einstein condensate type system (|1.2j) with Vi(a;) = — rfa 
in critical case N = 4. Note that A4 = 1. Then we have the following results. 

Theorem 1.2. Assume that N = 4, a = j3 = 2 and Ai, A 2 G (0, 1). Define 

. 1 f 1-Ai 1-A 2 (1-Ai)i(l-A 2 )f 1 

z/i := mm - < , , 5 5- > . (1-21) 

2\l-A 2 ' 1-Ar' (1 - Ai)S + (1 - A 2 )S / V ' 

Then for any v £ (0, Vx), il.20\) has a positive around state solution (u,,. v„) 6 P. 
which satisfies 

c v = J„{u v ,v„)^^(S{\i) 2 +S(\ 2 ) 2 ), asu^O. (1.22) 
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Theorem 1.3. Assume that N = 3, a + (3 = 2*, a > 2, (3 > 2 and X 1 ,X 2 G 
(0, A3). Then there exists V\ > such that for any v G (0,?i), /ias a 

positive ground state solution (u v ,v v ) G D, which satisfies 

Ju{u v ,v v ) -> i (5(Ai) 3 / 2 + 5(A2) 3/2 ) , a«i/->0. (1.23) 

Remark 1.1. (%) lljm - JJjm yield c v > ± max{S*(A 1 ) A '/ 2 , S^)^ 2 } /or 
i/ > small appropriately, that is, we obtain positive ground state solutions 
with energy above -A- max{S'(Ai) Ar / 2 , S 1 ^)^ 2 }, so the case min{a, /?} > 2 
is completely different from the cases studied in Theorem \l.lY f3). As we 
can see in the following sections, the case minjo;, f3} > 2 is much more 
complicated (see Theorem \2.1\ for example). Besides, if Ai = A2, then 
Theorem \1.2\ will be improved by Theorem \7.1\ in Section 7. 

(2) The case TV = 3 is much more tough than the case N = 4, and we can not 
give an accurate definition of v\ in Theorem \1.3\ as \1.21\) unfortunately. 
As we will see in the following sections, the idea of proving Theorem ] 1.2\ 
takes full use of the fact a = /3 = 2, and can not be used in the case N = 3. 
Meanwhile, the ideas of proving Theorem \1.3\ is quite different and more 
general. 

As we will see in Section 2, the radial symmetry of ground state solutions 
obtained in Theorem 11.11 is an easy corollary of the Schwartz symmetrization. 
However, the Schwartz symmetrization can not be used to prove the radial 
symmetry of ground state solutions obtained in Theorems II. 21 and II. 31 Here, to 
get the radial symmetry of solutions obtained in Theorems 11.21 and 11.31 we will 
use the moving planes method. Precisely, we have the following result. 

Theorem 1.4. Assume that N = 3 or N = 4, a + (3 = 2*, a > 2, j3 > 2, 
Ai,A2 G (0,Ajv) and v > 0. Then any positive solutions of WJm is radially 
symmetric with respect to the origin. Therefore, ground state solutions (u v ,v v ) 
obtained in Theorems \1.2\ and Theorems \1.S\ are radially symmetric. 

There are some other special cases, such as the case in which N = 3, 4, 5, 
1 < a < 2 < f3, a + /3 = 2* , X2 < Xi and v > sufficiently small, where we have 
no idea whether the ground state solutions exist or not. This remains to be an 
interesting open question. Here we can obtain positive solutions for these cases 
if v > is sufficiently small. Precisely, we have the following result, which plays 
a crucial role in the proof of Theorem 11.31 

Theorem 1.5. Assume that N > 3, Ai,A2 G (0,Ajv) and il.5\) hold. Then 
there exists v 2 > such that for any v G (0,^2], has a positive solution 

(u„, v u ) el, which is radially symmetric with respect to the origin and satisfies 

Ju{u v ,v v ) < i (S(\i)% +S(A 2 )£) . (1.24) 
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We should mention that, Abdellaoui, Felli and Peral PQ studied the following 
class of weakly coupled nonlinear elliptic equations 

-Au - Jjfau = u 2 *- 1 + uh(x)au a - 1 v 13 , x E R N , 
-Aw- = v 2 "- 1 + vh(x)Pu a v l3 -\ xeR N , (1.25) 
u, v G D^ 2 (R N ), u, v > in R N \ {0}. 

Note that if h(x) = 1, then (fl~25l) turns to be (|L4|) . For the case (|T3|) . they 
assumed the following condition on h(x) 

(Hi) h e L°°(R N ), h > 0, h ^ 0, h is continuous in a neighborhood of and 
oo, and h(0) = ]iEQ-\ x \->oo h(x) — 0. 

and then they proved some existence results of ground state solutions for (|1.25|) 
in case v > (see Section 4 in [1]). We should point out that, under condition 
(Hi), the Palais-Smale condition holds for energy level c with 

c<lmin{5(A 1 ) JV / 2 , S(X 2 ) N / 2 } , (1.26) 

(see p] Lemmas 4.1 and 4.3]), which plays a crucial role in obtaining ground 
state solutions in pQ. Therefore, problem Jj.^[ ) is completely different from 
$1.25)) . Moreover, there are no results about the existence of ground state 
solutions to (fi~25l) with energy above i max{5(Ai) JV / 2 , S(X 2 ) N/2 } in pp. It 
was only pointed out in [U Remark 4.6] that for the case where h(x) = 1, v > 0, 
a + /? = 2* and Ai = A2 = A, it is easy to construct, by a direct computation, 
positive solutions to (|1.4[) of the form ((f), aj)),c > 0. Remark that whether these 
solutions (<j>, c0) are ground state solutions is not known in [T] , and there are no 
any conclusions about (|1.4[) for the general case Ai 7^ A2 in pQ. 

The rest of this paper proves these theorems, and we give some notations 
here. In the sequel, we denote positive constants (possibly different in different 
places) by C, C\, C2, ■ • • , and B(x,r) := {y € R N : \x — y\ < r}. Denote 
B r := 5(0, r) for convenience. The paper is organized as follows. 

We give the proof of Theorem 11.11 in Section 2, where we will use the 
concentration-compactness principle from Lions ( [231 24]) and some ideas from 
[1 . The proof of Theorem II .21 is given in Section 3, where we will borrow some 
ideas from [33] and the authors' paper [P2] . 

In Section 4, we will prove Theorem 11.51 via a perturbation method, where 
we will use some ideas from Byeon and Jeanjean [5]. In order to construct a 
spike solution of the following nonlinear elliptic problem 

-e 2 Au + V{x)u = f(u), u e H 1 (R N ), 

for a general subcritical nonlinearity f(u) and sufficiently small e > , Byeon 
and Jeanjean [9] developed a new variational approach. We will mainly follow 
this variational approach to prove Theorem 11.51 Note that this approach can 
not be used directly, and we need some crucial modifications for our proof. For 
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example, we will define a special mountain-pass value a u , where all pathes are 
required to be bounded in D by a same constant which is independent of v. 
This special a„ is essential to our proof. Moreover, we should point out that 
the lack of compactness is the main difficulty because of the failure of the (PS) 
condition of (|1.4p . and especially because Zi,i = 1,2 are not compact in ID. 

In Section 5, we will prove Theorem ll.3l with the help of Theorem 1 1.5 1 Here, 
quite different ideas are needed comparing to those of proving Theorem 11.21 in 
Section 3. 

In Section 6, we will prove Theorem II .41 via the moving planes method. The 
moving planes method has been used by many authors to prove symmetry and 
monotonicity of positive solutions to various nonlinear elliptic problems, we refer 
readers to pjjl [Til H3 EEB] and references therein. 

Finally, by following some arguments from the authors' papers [T2l [13] , we 
will give some remarks for the special case Ai = A2, a — (3 = 2*/2 and v > in 
Section 7, where some uniqueness results about the ground state solutions will 
be obtained, see Theorems 17.11 and 17.21 In authors' papers [THEI], we studied 
the following Bose-Einstein condensation system for critical case 



Here, f2 c M. N (N > 4) is a smooth bounded domain and \i\ , /12 > (the special 
case N = 4 was studied in [TJ], and the general case N > 5 in 13 ). When v = 0, 
(|1.27p turns to be the well-known Brezis-Nirenberg problem (|S]). Thanks to 
the cerebrated idea from Brezis and Nirenberg $3j, we can show that the (PS) 
condition of (jl.27j) holds for some ranges of energy level (see |T21 [13] for details). 
Therefore, problem (|1.4j) is also completely different from (|1.27j) . Fortunately, 
some ideas of studying (|1.27|) in [TJl [13] can be used in this paper. 



Lemma 2.1. If c v is attained by a couple (u,v) € M v , then this couple is a 
critical point of J v , provided v < 0. 

Proof. This proof is standard. Let v < 0. Assume that (u,v) € M v such that 
c„ = J v (u,v). Define 




(1.27) 



2 Proof of Theorem 11.11 



2.1 The case v < 



G 1 (u,v) = \\u\\l- j (\uf+ua\u\ a \vf), 



Then there exist two Lagrange multipliers K\ , K2 £ M such that 
J' v (u,v) + K 1 G' 1 (u,v)+K 2 G' 2 (u,v) = 0. 




(2.1) 
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Testing (|2.1I) with (u,0) and (0, v) respectively, we conclude from (u,v) € AC 
that 

(2* - 2)|u||: + a(2 - a)\u\ f \u\ a \vA K x - a0\u\ ( f \u\ a \vA K 2 = 0, 

JR N J \JR N J 

(2* - 2)\v\%. + 0(2 - f \u\ a \vA K 2 - a p\ v \ ( f \u\ a \vA K x = 0. 

JR« / \JR N J 

Recall that > a|^|/ MJV |it| a |u|^ and > /3|H / ffiJ v M Q M^, we see from 
a + (5 = 2* that 

((2*-2)\u$+a(2-a)\u\ [ \u\ a \vA 

\ JR N J 

X ({2*-2)\v\f,+P{2-P)\v\ J \u\ a \vA > ( a p\u\ f \u\ a \vA . 

\ JW N J \ JR N J 

From this we deduce that K\ = K 2 = and so J' v {u, v) = 0. □ 
Lemma 2.2. Let v < 0. For any (u, v) G D urct/i u ^ and i> ^ 0, «/ 

/3 / \ 2* 



K'f) >a Q /^(Wj<l^) , (2-2) 



i/ien i/iere esisf ti > 0, si > 0, swc/i £/iai (tilt, Siu) G A/"„. 
Proof. For simplicity, we denote 

\u\ a \vf, A, = \\v\\l a) B 2 = \v\t. 



A 1 = || u || 2 Ai , B 1 = \u\ 2 2 :, C = \v\\ 



Recall the definition (|1.15l) of Af v , we see that (tu,sv) G M v for t, s > is 
equivalent to t, s > satisfy 



Ait 



2-a 



Bif-aCs , A 2 s 2 - i:s = B 2 s a - f3Ct a . 



(2.3) 



If C = 0, then it is trivial to see that (|2.3p has a solution (ti, Si) with t±,sx > 0. 
So we may assume that C > 0. Then the equation Ait 2 ~ a — Bit 13 — aCs^ is 
equivalent to 



s = g(t) 



Bit? - Ait 2 
aC 



V/9 



, Ai 



Therefore, it suffices to prove that 



Bit - Ait 2 ~ a 
c^C 



B 2 



Bit? - Ait 2 - a Y 



aC 



has a solution t > to- Note that (12.41) is equivalent to 



/(*) := A 2 



Bi - Ait 



2-/3 

2-2* \ —0- 



+ t 



2*-2 



(3C-B 2 



+ pCt a = (2.4) 



Bi-Ait 2 - T \ 13 
aC J 
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Note that ((2~2"j) implies that 

then it is easy to check that lim t \^ to f(t) > and lim t _> +00 f(t) = — oo. So 

there exists ti > t > such that f(ti) = 0. Let si = <?(ti), then si > and 

(tiu,siv) g A/1/. This completes the proof. □ 

Proof of Theorem ITTH-(l). Fix any v < 0. Recall (Tnty - (fL~T2"|) . it is easy to 
see that z* weakly in D 1 ' 2 (M JV ) and so (z 2 )* 9 weakly in L 2 */^(R W ) as 
/i — > +oo. That is, 

lim \u\ I (z{r(zlfdx = 0. 

fl^ + CO J RN 

Then (|2.2p holds for (zj,z 2 ) when /i > sufficiently large, and so there exist 
t^jS^ > such that (t^z^s^z 2 ) G A"„. Denote F M := \v\ J RN {z\) a (zffi dx. 
Then 



^(X^ = tl'SiX^-at^F,, slS(X 2 )^ = #S(A 2 )^ - (3t^F,. (2.5) 
Assume that, up to a subsequence, — > +oo as /i — >■ oo, then by /3(£ 2 — 



t 2 )S(Xi) 2 = a(s 2 * — s 2 )S(X 2 ) 2 we also have s M -> +oo. Then 



t 2 * -t\> , s 2 * - s 2 > -s 2 * for n large enough. 
Combining this with (|2.5[) we see that 

-M <2aS(Ai)-^F M -s-0, ( -S ) < 2pS(X 2 y^F fM -> 0, as ^ -> +oo, 



a contradiction. Therefore, and are uniformly bounded. Then by ()2.5[) 
and F M — >■ as [i — > oo, we get that lim /J _ s . +00 (i Al , s^) = (1,1). Note that 
(tnzl, s^z 2 ) € AC, we see from (|1.12l) and (I1T6P that 



Letting /z ->■ +oo, we get that c v < ^(^(Ai)^ 2 + S , (A 2 ) JV/2 ). On the other 
hand, for any (u, w) € A"„, we see from v < and (|1.17[) that 



< 



\u\ 2 'dx<S(\i)- 2 '' 2 \\u\\£, 



and so Hulllj > S(Xi) N / 2 . Similarly, \\v\\l 2 > S(X 2 ) N/2 . Combining these with 
HTJm . we get that c v > ^(^(Ai)^/ 2 + S{X 2 ) N / 2 ). Hence, 

C V = ±(S{X X ) N I 2 + S{X 2 ) N / 2 ). (2.6) 
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Now, assume that c v is attained by some (u,v) £ N Vl then (|u|, \v\) € N v 
and J,/(|u|, \v\) — c„. By Lemma |2.1[ we know that (|u|,|u|) is a nontrivial 
solution of (|1.4|) . By the maximum principle, we may assume that u > 0, v > 
in R N \ {0} and so J RN u a v^ dx > 0. Then 

\\u\\l < I | U | 2 *dx<5(A 1 )- 2 */ 2 || U ||r i , 

Therefore, it is easy to see that c v = J v {u,v) > ^■(5(Ai) Ar/2 + S'(A 2 ) Ar/2 ), which 
is a contradiction. This completes the proof. □ 



2.2 The case v > 

In this subsection, we let v > 0. Dehne 



inf J v {u,v), 



(2.7) 



where 



K ■= {(u,v)eD\ {(0, 0)} : J>, v)(u, v) = o}. (2.8) 



Note that M v C h!' v , so < c y . By (|1.17[) it is easy to prove that d v > 0. 
Moreover, it is standard to prove that 



c,= inf max J v (tu. tv) 

(u,u)eD\{(0,0)} *>o 



(m,-u)GD\{(0,0)} N 



nil + nil 



(f RN \ur+2*v\u\«\ v \p + \vry 



(2.9) 



Define E(u,v) := |Vu| 2 + |Vw| 2 - j^i\u\ 2 - y^M 2 an d F(u,v) 
2*v\u\ a \v\ 13 + \v\ 2 " for simplicity, then 



12* 



u) da; > (iVc'J 



I \7T 



R N 



F(u,v)dx\ , V (u, v) G 



(2.10) 



The following lemma is the counterpart of Brezis-Lieb Lemma ([7]) for (u, v), 
and the idea of its proof comes from [7] (see also [35J Lemma 1.32]). 

Lemma 2.3. Let D, C R N be an open set and (u n ,v n ) be a bounded sequence 
in L 2 (ft) x L 2 (il). If (u n ,v n ) — > (u,v) almost everywhere in Q, then 



lim / (\un\ a \v n f- \un-u\ a \v n -vf) dx = / \u\ a \vf dx. 
Proof. Fatou Lemma yields 



\u\ 2 < lim inf 



< oo, / \v\ 2 < lim inf 



\v n \ 2 < oo. 
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Recall that a, (3 satisfy (|1.5p . For any <zi, 02, 61, & 2 G and e > 0, we deduce 
from the mean value theorem and Young inequality that 



\a 1 +a 2 \ a \b 1 +b 2 f -^^f 

\a x + a 2 \ a - \ai\ a \h + b 2 f + \ ai \ a \h + b 2 f - \h\ 



< 



<C [(|oa| + Mr^CN + NW| + |ai| a (|&i| + NV^N] 



Ce l ' T (\a 2 f +\b 2 f 



<Ce [(M + M) 2 * + (|&i| + N) 2 * 
<Ce[\ ai f + \a 2 \ r +\b 1 \* + \b 2 f)+C i " (M~ ! 

Denote w n = u n — u and <7„ = v n — v, then 



i^n^i^-Krkni^-M a H^ 

-C £ (K| 2 * + | U | 2 *+|a„r + |tf* 

<\u\*\v\P + CS 1 - 2 ' (\uf +\vf) : 

and so the dominated convergence theorem yields da; — > as n — > 00. Note 
that 



\u n \ a \Vnf -\Un\ a \<Jn\ P -\u\ a \v\P 



we obtain 



< /= + Ce |w„ 



lim sup 

n— >oo Jo 



|tg>»|^wJ>nlM«|>| 



< Ce. 



Since C > is independent of e > 0, the proof is complete. 



□ 



The following lemma is the counterpart of Lions' concentration-compactness 
principle ([331 121]) for problem (TO)) . 

Lemma 2.4. Let G D be a sequence such that 



Define 



(u n ,v n ) — 1 (u,w) weakly in D, 
(u n ,v n ) — > (u,v) almost everywhere on M. N , 
E{u n — u,v n — v) — /i m </ie sense of measures, 
F(u n — u, v n — v) p m </ie sense of measures. 



/loo := lim limsup / E(u„,v n ) dx, 

R^oo „^oo J\ X \ >R 



(2.11) 
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lim limsup / F(u n ,v n )dx. (2-12) 

R-s-oc n-j.oo J\ X \> R 

Then it follows that 



Poo := 

R— s-oo 



IH| > (Nc' v )» ||p||£, (2.13) 
/ioo > (Nc'^pi, (2.14) 



limsup / E(u n ,v n )dx= / •») do; + ||/i|| + /i^, (2.15) 

limsup/ F(u n ,v n )dx= / u) da; + \\p\\ + p^. (2.16) 



z 2 

Moreover, if (u,v) = (0,0) and ||/z|| = (Nc v ) N \\p\\^ , then p and p are concen- 
trated at a single point. 

Proof. In this proof we mainly follow the argument of [35j Lemma 1.40]. First 
we assume (u, v) = (0, 0). For any h <E Cfi°(R N ), we see from (|2.10[) that 



E{hu n ,hv n )dx>{Nc' v ) N { \h\ 2 F(u n ,v n )dx) . (2.17) 

\JR N / 

Since u n 0, v n in Lf oc (R JV ), we have 

/ E(hu n , hv n ) dx — / |/i| 2 £'(u„, w„) da; — > as n — > oo. 

Jr n Jwl n 

Then by letting n — > oo in (|2.17[) , we obtain 

\h\ 2 dp>(Nc'^([ \hfdpY , (2.18) 



that is, (|2~T31) holds. 

For R > 1, let Vfl € C 1 ^) be such that < ^ < 1, Vfl(^) = 1 for 
\x\> R+l and ^(a;) = for \x\ < R. Then we see from (f2~T0|) that 



E(ip R u n , i])RV n ) dx > {Nc' u )~ ( / \ip R \ 2 F(u n ,v n )dx 
Since u n 0, v n in Lf^R^), so 
limsup / |i/'fl| 2 £ , (un; ^n) da; 

> (JVcJ,)* limsup (/ |^| 2 >K,w„)da;) . (2.19) 



n— >oo VJIR-^ 
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Note that J\ x \> R+1 F(u n ,Vn) < J r n \^r\ t F(u n , v n ) < J^ R F(u„,v n ), so 

Poo = lim limsup / \^r\ 2 F(u n ,v n )dx. (2.20) 

R-too n^oo J r n 



fl-Kx> rl _J.oo J R N 

On the other hand, 



limsup / \ip R \ 2 E(u n , v n ) dx 
n— >oo Jr n 

= lim sup / E(u n , v n ) dx + limsup / \4>r\ 2 E(u ni v n ) dx 

n-^co J\ x \>R+l n-s-oo J R<\x\<R+l 

— limsup / E(u n , v n ) dx + limsup / |"0-r| 2 (I | 2 + |Vw„| 2 ) dx 

ri^oo J\ X \>R+1 n->oc J R<\x\<R+l 

> lim sup / E(u n ,v n )dx. 

n^oo J\x\>R+l 



Letting R — ¥ oo we see that p^ < lim^^oo limsup,^^ J„ N \ipR.\ 2 E(u n , v n ) dx. 
Similarly, 



limsup / \ip R \ 2 E(u n ,v n )dx 

n— >oo JRN 

= limsup / E(u n , v n ) dx — liminf / 

n^oo J\x\>R n-Hx> J R 

< lim sup / E(u n ,v n ) dx. 

n-s-oo J\x\>R 



(1 - \ip R \ 2 )E(u n ,v n )dx 

R<\x\<R+l 



Letting R — >• oo we see that p^ > lim^^oo limsup,,^^ J RN \ipR\ 2 E(u n , v n ) dx. 
Hence 

Poo = lim limsup / \ipR\ 2 E(u n ,V n ) dx. (2-21) 
Then flOD follows directly from (j^TB)) . (|2T2T)1) and (JMH). 

_2_ 2 

Assume moreover that \\p\\ = (Nc' u ) N \\p\\^ . Then by Holder inequality 
and (|2.18[) . we have 

\hfdp< {NdJ-T^Wup** [ \hf dp, VheCZ°{R N ). 

From this we deduce that p = {Nc' u )~ \\p\\ tt=2 ^. So p — {Nd v )%\\p\\~% p, 
and we see from (|2.18p that 



\\p\\^ / \h\ 2 dp) < \h\ 2 dp, V^Ql"). 
\JR« / 7r« 

That is, for each open set Q, we have p(n)2*p(R N )N < p (fi). Therefore, p is 
concentrated at a single point. 
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For the general case, we denote cu n — u n — u and a n — v n —v, then (u) n ,a n ) 
(0,0) weakly in B. From Brezis-Lieb Lemma ([7]) and Lemma [2.31 we obtain 
for nonnegative h £ Co(Ni N ) that 

hE(u,v)dx = lim / hE(u n ,v n )dx— / hE(uj n , o~ n ) dx 

rwoo \J BN ' l„ N 



SO 



hF(u,v)dx — lim / hF(u n ,v n )dx — / hF{uj n ,a n )dx 



E(u n , v n ) — 1 u) + /i, F(u n , v n ) — 1 w) + p, in the sense of measures. 

(2.22) 

Inequality (|2.13[) follows from the corresponding one for (oj n , cr n ). From Brezis- 
Lieb Lemma ([7]) and Lemma 12.31 again, it is easy to prove that 



(loo := nm limsup / E(uj n ,<J n ) dx, 

R-^oo n -HX J\ X \> R 

Poo := lim limsup / F(ui n ,a n ) dx, 

R-^oo n ^oo J\ X \ >R 



<\x\>R 

and so inequality (|2.14l) follows from the corresponding one for (w n ,a n ). For 
any R > 1, we deduce from (|2.22l) that 

limsup / F(u n ,v n ) 
= limsup(/ \tp R \ 2 * F{n n ,v n ) + (1 - \ip R f)F(u n , v n ) ) 

n^oo \JRN 7 R N / 

= lim sup f \yj R \ r F(u n ,v n )+ [ (l-\^ B f)F{u,v)+ [ (l-\^ R f)dp. 



Letting R -> oo, we see from ([2~2U1) that (|2~To) holds. The proof of ([2~T5|l is 
similar. This completes the proof. □ 

Lemma 2.5. Let v > 0. TTien j /ias a solution (u, «)£B\{(0, 0)} (maybe 
semi-trivial), such that J u (u,v) = c' v and u,v > are radially symmetric 
with respect to the origin. Moreover, if d v < fa min{5(A 1 ) Ar / 2 , S(X 2 ) N/2 }, then 
(u,v) £ D is a positive ground state solution of jl.4]) , and c„ = d v = J u (u,v). 



Proof. For (u, v) £ M' v with u > 0, v > 0, we denote by (u*,v*) as its Schwartz 
symmetrization. Then by the properties of Schwartz symmetrization (see |21) 
for example), we see from Ai, A2, v > that 



(\Vu*\ 2 + \Vv*\^-^\u\ 2 -A\v*\ 2 )< [ (\u*f+2*v\u*\ a \v*f+\v 

I \X\ \X\ J R N 

Therefore, there exists < t* < 1 such that (t*u*,t*v*) £ N' v , and then 

Mt*u*,t*v*) = ^(n 2 (\\u*\\i i + \\v*\\i 2 ) 
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< ^(II«IIa 1 + \Hl 2 ) = Ju(u,v). 



(2.23) 



Therefore, we may take a minimizing sequence (u n ,v n ) £ M' v of c' v such that 
(u n ,v n ) = (u nl v n ) and Jv(u n ,v n ) — > d v as n — > oo. Define the Levy concentra- 
tion functions 



Q n [R) := sup / F(u n ,v n ) dx. 

2/£R« JB(y,R) 



Since u n ,v n > are radially symmetric non-increasing, one has that Q n (R) 
jb(o r) Fi^ni Vn) dx. Then there exists R n > such that 



Qn\Rn) 



F(u n ,v n )dx=- F(u ni v n )dx 



B(0,R n 



Define 



( iV-2 JV-2 
R n 2 U n (R n x) 7 R n 2 V n (R n x) 



Then by a direct computation, we see that {u n ,v n ) £ M' v , J^(u n ,v n ) —} d v , 
Un,Vn > are radially symmetric non-increasing, and 



B(0,1) 



F(it n , u n ) = - / F(u n ,v n )dx = sup 



yeR" JS(!/,1) 



F(u n ,v n )dx. (2.24) 



From (|2 . 23[) we know that (u n ,v n ) are uniformly bounded in B. Then passing 
to a subsequence, there exist (u, v) £ B and finite measures fi, p such that (|2.11[) 
holds. Define /UooiPoo a s in (|2.12|) . then by Lemma [2~4l we see that (|2.13[) - (|2.16[) 
hold. Note that 



/ F(u n , v n ) dx — > iVcJ,, as n — > oo, 
Jr n 

we conclude from ^ZJSty - fiLWty and (|2TTU|) that 
Nc' v = / F(u,u) cte + ||p|| + poo, 



(2.25) 



#4 > (tvc'j- 



Therefore, J RN F(u,v) dx, \\p\\ and poo are equal either to or to Nd v . By 
(I2.24|) - (I2.25|) . we have p^ < \Nc' v , so p^ = 0. If ||p| = Nd v , then one has 
that J RJV F(u,v)dx = and so (u,v) = (0,0). Moreover, since \\p\\ < Nc' v , we 

deduce from (f2~T3|) that ||p|| = (Nd u )^ \\p\\& . Then Lemma l2~4l implies that p 
is concentrated at a single point z, and we see from lj2.24jl - (|2.25Jl that 



\nc! v = lim sup / F(u n ,v n )> lim / F(u n ,v 7 

2 n ^°°ym N JB(y,l) n ^°°JB(z,l) 



) = \\PI 
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a contradiction. Therefore, J RJV F(u,v)dx — Nc' v . Since \\u\\ 2 Xi + \\v\\\ 2 < Nc' h 
we deduce from ([2~T0)) and (pf25l) that 




that is, (u n ,v n ) — > (u,v) strongly in D, (u, v) € M' v and J v (u, v) — d v . Recall 
that c' v > 0, so (u,v) ^ (0,0). By the definition (|2.5[) of M' v and using the 
Lagrange multiplier method, it is standard to prove that J' v (u,v) — 0, so (u,v) 
is a solution of (ll.4[) . Moreover, u,v > arc radially symmetric. 

Now, assume that c„ < -A> min{5(Ai) JV / 2 , 5 , (A 2 ) Ar / 2 }, then it is easy to prove 
that both u ^ and v ^ 0, that is, (it, w) G J\f u , and so J„(w, u) = d v = c v . 
Hence, (u,v) is a ground state solution of (jl.4l) . By the maximum principle, 
u, w > in R N \ {0} and are radially symmetric. This completes the proof. □ 

Since (^,0) and (0,z 2 ) belong to M' v , so c' u < min{S(\i)% , S(X 2 )%} 
always holds. However, the following result says that the conclusion c' v < 
min{S'(Ai)"^", S(X 2 )%} can not always hold unfortunately. 

Theorem 2.1. Assume that a, ft > 2, then there exists v > such that for all 
v G (0, v) there hold 

Moreover c' v is achieved by and only by 

UQ,±4), M>0, i/Ai<A 2 , 
U±^,0), ^>0, z/A 1 >A 2 , 
((0 t ±4), (±zj v 0), n>0, if\i = X 2 . 

Proof. Thanks to Lemma |2~31 this proof is completely the same as that of [1, 
Theorem 3.4], and we omit the details here. □ 

Proof of Theorem ITTTI (2)-(3). Let v > 0. By Lemma [231 and Theorem HO 
we know that, we have to require further assumptions on a, j3 and v to obtain 
positive ground state solutions with energy below min{S'(Ai)~, 5(A2)~}. 
Denote 

An — A2 , Ajy — Ai 

dl : " a — T' d2 := a — r- 

An — Ai Ajv — M 

Recall (|1.18j) . we let v > u . Then 

l+max{di,d 2 } < (2 + 2V)*. (2.26) 

Without loss of generality, we may assume that Ai < X 2 . Then (| 1 . 1 1 1) yields 
S(X 2 ) < S(Xi). By Hardy inequality QUfy we have ||tt||^ < <felMl! 2 for all 
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u e D 1 ' 2 (R N ). Then we deduce from (iLTTj) . 



and (gHHl that 



< 



< 



N 
1 

iV 



(/ RN |Z2|2* + 2V|Z2|2* + | Z 2| 



Z 2 II 2 



(/r 



^2 12* 



imin{5(Ax)^,5(A 2 )^}. 



Hence, conclusion (2) follows from Lemma [ 

Repeating the proof of [TJ Theorem 2.2 (iii)-(iv)] with minor modifications, 
we can show that, if a < 2, then for all /j, > 0, (0, z 2 ) is a saddle point for 
J„ in AC, and so d v < ±S(X 2 ) N/2 ; if /3 < 2, then for all n > 0, (a£,0) is 
a saddle point for J v in Ml, and so c'^ < j^S{X\) N / 2 . Meanwhile, by 
we see that, Ai < A2 implies S(Xi) > S(X2) and Ai > A2 implies S'(Ai) < 
5(A2). Then under any one conditions of (Ci), (C2) and (C 3 ), we have that 
c' v < jj min{S'(Ai) Ar / 2 , S(X2) N ' 2 }, and so conclusion (3) follows from Lemma 
12.51 This completes the proof. □ 



3 Proof of Theorem [L2l The case N = 4 

In this section, we assume that N — 4, a = j3 = 2 and Ai,A2 G (0,1). By 
Theorem 12.11 we know that, the ideas of proving Theorem 11.11 can not be used 
here, and we need to use a different approach, which is much more complicated. 
This approach will take full use of the fact a = /? = 2, and so can not be used 
in the case N = 3 unfortunately. 



3.1 The special case Ai = A2 = 

Consider the following problem 

' -Au = u 3 + 2vuv 2 , x G R 4 , 

-At; = v 3 + 2isvu 2 , x G K 4 , (3.1) 
u, v G D 1,2 (M i ), u, w>0inR 4 . 

For e > and y G K 4 , we consider the Aubin-Talenti instanton [HJ [32] 
U e ,y G L>^ 2 (R 4 ) defined by 
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Then U £ , y satisfies — Au = u 3 in R 4 and 



WU £ J 2 dx = \U £ J 4 dx = S 2 . (3.3) 



Furthermore, {U £ . y : e > 0, y E K 4 } contains all positive solutions of the equa- 
tion —Ait = u 3 in M 4 . Note that (|3.1I) has semi-trivial solutions (U £>y ,0) and 
(0, U e>y ). Here we are only interested in nontrivial solutions of (|3. II) . which can 
be found as nontrivial critical points of the C 2 functional L v : D — > M, where 

£,,(«,«) = J (||u|| 2 + |M| 2 )-i / (u 4 + 4^ 2 w 2 + w 4 ) . (3.4) 

Definition 3.1. FFe say a solution (uo,vq) of liS.l]) is a ground state solution if 
(uo, fo) is nontrivial and L v {uq, Vq) < L v {u, v) for any other nontrivial solution 
(u,v) of fSJ$. 

Define the general Nehari manifold of fl3.4[) as 

M v := \(u,v) G D : u 0,v # 0, / \\7u\ 2 = [ (u 4 + 2i/u 2 w 2 ), 

^ JR 4 JR 4 

|Vw| 2 = / (w 4 + 2^ 2 w 2 )}. 

JR 4 J 

Then any nontrivial solutions of (|3.1[) has to belong to M. v . Similarly as Af v , 
we see that M v £ 0. We set 

m,:= W M«,«)= ( inf J / (|V W | 2 + |Vw| 2 ) dx. (3.5) 

By Sobolev inequality (|1 . 13|) . it is easily seen that m v > for all v. Moreover, if 
(tto, fo) is a nontrivial solution of (|3.1[) satisfying L u (uq, vq) = m„, then (ito, wo) 
is a ground state solution. Then we have the following result, which will play 
a crucial role in the proof of Theorem 11.21 Part of this result comes from the 
authors' paper [12]. 

Theorem 3.1. Let v > 0. 

(1) If v =£1/2, thenforanye>0,y el 4 , {{l+2v)~ 1 / 2 U e >y , {l + 2v)- 1 / 2 U £tV ) 
is a positive ground state solution of \3.1)) , with 



m v = L, 



((1 + 2 v )-^ 2 U e , y , (1 + 2v)-i/ 2 U £ , y ) = 2{l l +2u) S 2 - (3-6) 



Moreover, the set {(1 + 2v)- 1 / 2 U s , y , (1 + 2u)- 1 / 2 U e>y ) : e > 0,y G R 4 } 
contains all positive ground state solutions of \3.1]) . 

(2) If v = 1/2, thenfor any e > 0, y G R 4 , (9 e (0,tt/2), (sin 0Z7 6)I „ cos 0Ef e , v ) 
is a ground state solution of H3.1]) , and mi/2 = \S 2 . Moreover, the set 
{(sin0 U £:V , cos6U ejV ) : e > 0,y G R 4 ,0 £ (0,7r/2)} contains all positive 
ground state solutions of 13.1]) . 
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Proof. (1) Let v > and v ^ 1/2. Then this result follows directly from [T2l 
Theorem 1.5 and Theorem 4.1]. 

(2) Let v = 1/2. Firstly, note that for any 6 € (0,tt/2), (sin 6»t/ e , y , cos9U e , y ) 
is a positive solution of (|3.1j) when f = 1/2, so 

m 1/2 < L 1/2 (sm9U e , v , cos8U e , y ) = ^S 2 . (3.7) 
Secondly, take any (it, v) £ M.i/ 2 - If 

( [ mV dx) = [ u 4 dx [ v 4 dx, (3.8) 

VjR 4 / JR 4 JR 4 

then by Holder inequality, we may assume that v = Cu for some constant C/0. 
Recall the definition of M. v , we see from ()1 . 13[) that 

/ |Vit| 2 = (1 + C 2 ) f u 4 <(1 + C 2 )S- 2 ( [ \Vu\ 2 

JR 4 JR 4 \JR 4 

so J R4 |Vu| 2 > (1 + C 2 )- 1 ^ 2 and then 

^1/2 («) = L 1/2 (u, Cu) > ^S 2 . 
If ([378]) does not hold, then 

/ u 2 i> 2 I < / u 4 dx v dx, 

\JR 4 / JR 4 il 4 

and it is easy to prove that for any v £ (0, 1/2), there exist t v , s u > such that 
(y/Uu, y/s^v) € Mi, and (t v ,s u ) —> (1, 1) as f — s- 1/2. This implies that 

L\/ 2 {u,v) = lim L u {\ft^u,JJ^v)> lim m„ = ^-S* 2 . 

Therefore, for any (u, w) g 1/2, we have Li/2 (w, w) > jS 12 , and so > |5 2 . 
Combining this with p. 71) . we see that (wo.6U ev , cos8U E:y ) is a ground state 
solution of (|3.1j) when ^ = 1/2. 

Now assume that (it, v) is any a positive ground state solution of (|3.ip when 
^ = 1/2. Then we deduce from (jl . 13[) that 

S\u\ 2 < \\u\\ 2 = \u\ 4 + [ u 2 v 2 dx < \u\i + \u\l\v\l 

JR 4 

that is, \u\ 2 + \v\ 2 > S. Meanwhile, since L 1 / 2 (it,-u) = m 1 / 2 — jS 2 , we have 

S 2 = \\u\\ 2 + \\v\\ 2 >S\u\ 2 + S\v\l 
that is, \u\l + \v\l < S. So \u\\ + \v\\ = S, that is, 

S\u\t = \\uf = \u\t+ I u 2 v 2 = \u\t + \u\l\v\l 

JR 4 
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First this means that v — Cu for some C > 0. Secondly, combining (jl.l3[) with 
||u|| 2 = S\u\l, it is well known that u = G\XJ e ^ y for some C\ > 0, e > and 
y £ M. 4 (see [35] for example). Hence, (u,v) = (C\U e ^,CiU e>y ) for some 
Ci,C 2 > 0. Then L 1/2 (u,v) = \S 2 yields that C\ + C\ = 1. Therefore, there 
exists 9 £ (0,7r/2) such that C\ = sin# and C 2 = cos 6. This completes the 
proof. □ 



3.2 The general case Ai, A 2 G (0, 1) 

Recall the definition (|1.2ip of v\, we have the following important energy esti- 
mate, and the idea of the proof comes from the authors' paper [12] . 

Lemma 3.1. For any v £ (0, there holds 

c u < min jisXAi) 2 + jS(X 2 ) 2 , m t 
Proof. Define 

G(u,v) := ( „ k* U \ d * toJpMdx \ (39) 
v ' \ 2v J R4 u A V A dx J M4 v^dx J v ' 

When det G(u, v) > 0, the inverse matrix of G(u, v) is 

r<-V \ - 1 ( Jr 1 ' 4 ^ -2v f Ri u 2 v 2 dx \ , . 
G [U ' V) det G(u,v) { -2v^u 2 v 2 dx J Ri uUx )' (6AU > 

Assume v £ (0, v\). Obviously, one has that 2i> < 1 and so det G(z\, z 2 ) > 0. 
Recall that \\z\\\\ t = \z\\\ = 4S(A l ) 2 ,i = 1,2, we see that (y%z\, y/soz 2 ) £ Af v 
for some to > 0, so > is equivalent to 



to 

so I — ■ - , 



G-\zlzl) 



'1 14 
v 2|4 



_ 1 f - 2,J &i (zl) 2 (z 2 ) 2 ) \ / \ 

" detG^z 2 ) ^ |z 1 1 |l(|z 1 2 H-2^ R4 (z 1 1 ) 2 (z 1 2 ) 2 ) J > J ' 

Meanwhile, (1TTTT1) yields S x , = (1 - A;) 374 -?, and so we deduce from (TOT]) that 

2f < mm < , > < mm < — - — - , — ; — - 

I-A2' 1-ArJ " \5(Aa)' 5(Ai) 



Then 
2v 



(z x \ 2 (z 2 \ 2 <min f S ^ 5(A2) 





1 2|2 . J 


V1 4 






Pl| 4 = mm | 


1 1 u ' 


i 2 2 r 
pi 1. 



So (l3~TTj) holds and (Vto~zJ, ^z 2 ) £ M v for (t , s Q ) defined in (|3"TLj) . Then 

cu < Ju(VT zl^o4) = ^\\z\\\ + S i\\z 2 \\l 2 
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< <0 



^u 2 



T / ((^i) 4 + 2^i) 2 (^) 2 )+T / fe) 4 + 2^)W) 

4 V 7 4 JR* V 7 

= Jll*illL + l\\4\\l = \s(\ 1 f + \s(\ 2 f. 

Hence c„ < ^S^Ai) 2 + jS , (A 2 ) 2 . It remains to prove c v < m v . Take y e R 4 
such that | j/o | = 2. Let V € Cg°(B(y , 1), ffi) be a function with < iji < 1, 
V-> = 1 for x G B(2/o, 1/2). Recall E/ ElI/0 in (|3.2j) and (|3.3j) . we dehne f/ e := xjjU £} y . 
Then by [5] or [351 Lemma 1.46], we have the following inequalities 

/ \VU £ \ 2 = S 2 + 0(e% f \U £ \ 4 = S 2 + 0(s 4 ), 
/ K <fa > 5 / l^l 2 > Ce 2 | lne| + 0(e 2 ), 

J»4 |x| 2 9 iB(y ,l) 

where C is a positive constant. Recalling that Ai, A2 > 0, we have 

MViu s , VSu.) = \t I (|w £ | 2 - ^ 2 u 2 ) + \ s I (|w £ p 

- I(t 2 +4^s + s 2 ) / L/ £ 4 

<i(t + s) (S 2 -Ce 2 \lne\ + 0{e 2 )) 
-- A (t 2 + Avts + s 2 )(S 2 + 0{e A )). (3.12) 

Denote 

A £ = S 2 - Ce 2 \ lne| + 0(e 2 ), B e = S 2 + 0(e 4 ), 
then < A £ < B £ and A £ < S 2 for e > small enough. Consider 

f £ {t, s) := + «) - ^(£ 2 + 4i/t« + s 2 ), 

then it is easy to see that there exists t £ , s £ > such that 

fe(t £ , s £ ) = max f £ (t, s). 

t,s>0 

By mM** s )\(t e ,s s ) = ^/e(*) s )l(t £ , Se ) = 0, we see that 

* - As 

Then it follows from and (glgl that 

maxJu{ytU e ,y/aU e ) < max f e (t, s) = f e (t e ,s e ) 

i,s>0 t,s>0 
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1 A 2 1 

2(1 + 1v) B e < 2(1 + 2v) 6 
S 2 

< ^ — ^ j = m u holds for e small enough. (3.13) 



Similarly as above, we have det G(U e , U e ) > 0. Moreover, (\/i e U E , \flT E U e ) € 7V„ 
for some t e > 0, s e > is equivalent to 

te\_ \U e \i ( WUeWl-MUtWi, \ . ( 



detG(C/ £ ,f/ £ ) V l|C4lll 2 -2Hl^lli y " V 
On the other hand, by (| 1 .6[) we have 



> n • ( 3 - 14 ) 



\Ue\\l 



2v\\U e \\l a =(1 - 2u) f |Vf/ £ | 2 - (Ax - 2v\ 2 ) f ^- 

JR 4 Jl 4 Fl 

>(l-2i/) / i% _ (A x - 2i/Aa) ' 



2 



^>0. 

|x| 2 



(1 - Ax) - 2i/(l - A 2 ) 



Similarly, \\U e \\l 2 - 2v\\XJ s f^ > 0. Hence, (0H holds and (Vt e U £ ,VT £ U s ) € 
M, for (i e ,S e ) defined in QSTty . Then we see from (|3. 13|) that 



c v < J u ( V%U E , \/%U £ j < max < m„. 

\ / t,s>0 

This completes the proof. □ 

Lemma 3.2. Assume that v € (0, z/x). XTien t/iere exisi C2 > C\ > sztc/i that 
for any (u,v) € Af v with J v (u,v) < ■jS'(Ai) 2 + jS(\2) 2 , there holds 



Ci< [ u 4 dx, [ v A dx< C 2 . 
Jr 4 Jr 4 



(3.15) 



Proof. Take any (u,v) G M v with J v (u,v) < jS'(Ai) 2 + \S(X 2 ) 2 . By (fTTTf 
and Holder inequality, one has 

S(\i)\u\l<\\u\\ 2 Xl = [ (u i + 2vu 2 v 2 ) < \u\\ + 2v\u\ 2 \v\l 
Jr 4 

S(X 2 )\v\ 2 < \\v\\ 2 2 = [ (« 4 + 2vu 2 v 2 ) < \v\i + 2v\<\v\l 



Therefore, there exists C 2 > such that J Ri u 4 , L 4 v 4 < C 2 . Moreover, 

\u\\ + 2v\v\\ > 5(Ai), (3.16) 

2v\u\ 2 + \v\ 2 4 >S(X 2 ), (3.17) 

S(Ai)|«|2 + S(A 2 )M 2 < 5(A0 2 + S(A 2 ) 2 . (3.18) 
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Recall that S(Xi) = (1 - \i) 3/4 S. Since u G (0,^) and 1/1 is denned in ([L"2"T]) , 
by (|3~T6|) and (pTT8l) we have 



_> S(Ai)S(A 2 )-2i/(S(Ai) 2 + S(A 2 )^ 



sw-W.) >0 " (3 - 19) 

and by (f3TT7|) and (|3~18| we have 

g(A 1 )^(A 2 )-2 ; ,(5(A 1 ) 2 + 5(A 2 ) 2 ) 
|W ' 4 " 5(A0-2^(A 2 ) > °- 

This completes the proof. □ 

The following lemma is motivated by |33j , and some ideas of the proof come 
from (30]. 

Lemma 3.3. Assume that v G (0, V\). Let (u n ,v n ) G M v be a minimizing 
sequence of c„, and (u n ,v n ) (0,0) weakly in D. Then for any r > and 
for every e € (— r, 0) U (0, r), i/iere exists p G (e,0) U (0,e) swc/i i/iai, Mp to a 

eitfier / (|Vu„| 2 + |Vv„| 2 ) -> or / (|Vw„| 2 + |Vw„| 2 ) 0. (3.20) 

Proof. Without loss of generality, we only consider the case e G (0, r) (the 
proof for the case e G (— r, 0) is similar). Since G M v is a minimizing 

sequence of c u , then (ti n ,u n ) are uniformly bounded in D. Moreover, by Lem- 
mas [XT] and [XU we may assume that (u n ,v n ) satisfies (|3.15p for all n G N. 

Step 1. We prove (|3T20|) by further assuming that J' v (u n , v n ) — > as n — > oo . 

In the following, some arguments are borrowed from [30j (see also |31[ Lemma 
III. 3. 3] or [33, Proposition 5.2]). Denote § as the unit sphere of K 4 . Since 

f +£ dp f (\Vu n \ 2 + |Vu„| 2 ) = f (|Vu„| 2 + |Vi;„| 2 ) 

is bounded, we can find p £ (0, e) such that 

(\Vu n \ 2 + \Vv n \ 2 ) < - f (|Vu„| 2 + \Vv n \ 2 ) 

(r+p)S £ Jr<\x\<r+e 

holds for infinitely many n's. Therefore, as H 1 ((r + p)S) is compactly embedded 
into 7? 1 / 2 ((r + p)§), up to a subsequence we can assume that u n — > u, v n — > 
v strongly in _ff 1 / 2 ((r + p)8). On the other hand, by the continuity of the 
embedding H 1 (B r+p ) <-t iJ 1//2 ((r + p)S) and by the weak convergence to (0,0) 
of («„, v n ), we deduce that (u, v) — (0, 0), that is, u n — > and v n — > strongly 
in i? 1 / 2 ((r + p)E>). Let Wi. n , i = 1,2 be the harmonic functions satisfying 

Aiui,„ = in B r+e \ B. r+p | Akj 2 ,„ = in B r+p \ B r _ £ 
tui^ = on (r + e)S , < u> 2i „ =0 on (r- e)S , (3-21) 
i"L n = i n on(r + p)§ I W2,« = «n on (r + p)S 
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and let crj„, i = 1, 2 be the harmonic functions satisfying 



A<7i,„ = in B r+£ \ B r+p 
oi^n = on (r + e)S 
fl,n = "n on (r + p)S 




= in B,. +p \ B r _ £ 
on (r — e)S 
v n on (r + p)S 



(3.22) 



By the continuity of the inverse Laplace operator from H 1 ^ 2 (dfl) to H 1 ^), it 
follows from the above discussion that w\, n — > 0, o"i )n — > strongly in 7J 1 (i? r _|_ e \ 
-B r +p) and W2, n — >• 0, 02,71 - ^ strongly in H 1 (B r+p \ B r _ e ). Define 



(3.23) 





u n (x) 


if ,T e s r+p 




"l,n(l) — < 




if X € -B r +e 


\ B r+p 






elsewhere, 








if ,X € Br+p 




^i,n(a;) = < 




if x e B r+S \ B r+P 







elsewhere, 






'» 


if x G £? r _ e 




U2,n{x) = < 


W 2 ,n 


if x E B r+p \ B r - e 






elsewhere, 








if x G B r _ e 




V2,n(x) = < 




if x G B r+P \ B r _ s 




,w„(x) 


elsewhere. 




Then it is easy to see that 










Iwi^IlL 


+ ll^nllAj " 


f-o(l), 


\\Vn\\l 2 = 


"l.nllL 


+ Kn||A 2 + 


-0(1). 


Moreover, we can easily obtain 








J' v (u lin ,v 1>n )(ui in ,0) = 


= J' v ( U n, 


Wn)(wi,„,0) - 


Ml) = 


jU u l,n,Vl,n)(0,Vl, n ) = 




Un)(0,«l,n) "1 


-o(l) = 


J' 1/ (u2, n ,V 2 ,n)('U>2,n,0) = 




V n )(u 2 ,n,0) ~ 


r-0(l) = 


J'v(u2,n,V2,n)(Q,V2,n) = 




V n )(0,V 2tn ) ^ 


-o(l) = 



Then we claim that 



(3.24) 



(3.25) 



(3.26) 



(3.27) 
(3.28) 



(3.29) 
(3.30) 
(3.31) 
(3.32) 



either lim (|K„|| 2 + |K„|| 2 ) = or lim (||u 2 , n || 2 + ||« 2 ,„|D - 0. (3.33) 
In fact, if (13.33[) does not hold, then up to a subsequence, 



both lim (||wi,„|| 2 + ||^i,„|| 2 ) > and lim (||u 2 ,„|| 2 + lk 2 ,„|| 2 ) > 0. (3.34) 
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We have the following several cases. 

Case 1. Up to a subsequence, both lim ||ui,„|| 2 > and lim ||i>i, n || 2 > 0. 

n— >oo ' n— >oo ' 

Since norms || • ||a;,* = 1,2 are equivalent to || • ||. and (I3.29[l - (|3.30p yield 

ll^l.nllL =W, n \l + ^ I + (3.35) 

JR 4 

IK„|| 2 a 2 - \vi, n \i + 1v I + o(l). (3.36) 

JR 4 

Hence, both liminf |wi n || > and liminf \v\ n \\ > 0. Since 2v < 2v\ < 1, so 

n— ^oo ' n— too ' 

by Holder inequality we have 



lim inf 

n— »oo 



«1, 



\t\vi, 



2v 



> 0. 



Combining this with (I3.35|) - (I3.36|) . it is easy to prove that there exist t n , s n > 
such that (\/t^ui, n , y/s^vi, n ) £ M v and (t n ,s n ) — > (1,1), and so we conclude 
from ([3T?Tj) - ([3^5]) and (pTMj) that 

c„ = lim J v (u n ,v n ) = lim 7 (||m„||;L + IKIlL) 

= lim ^(tnllui.nll^ + s nlKn|lL) + lim 7 (II M 2,n || A x + || «2,™ || A 2 ) 

> lim T^nlK.nllAj + s "IK~IIaJ 
= lim J u (Vt^ui n , v/i^^i, n ) > c„, 

n— ^oo 

a contradiction. So Case 1 is impossible. 

Case 2. Up to a subsequence, lim \\m n || 2 = and lim ||fi. n || 2 > 0. 

n— >oo ' n— >oo 

Then (j3T36|) yields that 

\\vi,n\\l 2 = + "(I) < S(A 2 )- 2 ||« 1)n ||l 2 + o(l), 

and so lim |K Jl? > S(X 2 ) 2 . By (|5T5)| we have 

liminf ||ii2 n || 2 = liminf |jM„|| 2 — lim ||ui.„|| 2 > 0. 

n— )-oo ' n— »oo n— >oo ' 

If up to a subsequence, lim n _^ QO ||v2.n|| ^ Oi then we can get a contradiction 
just as Case 1. Therefore, limn-,.^ ||v2,n|| 2 = 0. Then similarly as above, we 
can deduce from ([3"3T]) that lim ||u 2 J? > S'(Ai) 2 . Then 

n— >-oc 1 

c v = hm ^(IKH^ + IKHaJ 

n— ¥oo 4 

= hm i(h 2 ,„|| 2 Al + Wv^Wl) > - (5(A!) 2 + S(A 2 ) 2 ) , 

n— too 4 4 
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a contradiction with Lemma 13.11 So Case 2 is impossible. 

Case 3. Up to a subsequence, lim ||ui,„|j 2 > and lim \\vi n \\ 2 = 0. 

n— J-oc 1 n— ^oo ' 

By a similar argument as Case 2, we get a contradiction. So Case 3 is 
impossible. 

Since none of Cases 1, 2 and 3 is true, we see that (|3.34p is impossible, that 
is, (13.33)) holds. Recall the definition of (ttj jn , v i}n ), (|3. 201) follows directly from 
(|3.33p . This completes the proof of Step 1. 

Step 2. We prove (13.201) without assuming that J' v (u n ,v n ) — > as n — > oo. 

By the Ekeland variational principle (see [31] Theorem 5.1] for example), 
there exists a sequence {(u n ,v n )} £ AC such that 

^(«n,?n)<^K,fn), || (u n , V n ) - (u n , V n ) \\ < - , (3.37) 

n 

J v (u,v) > J v {u n ,v n ) \\(u n ,v n ) - (u,v)\\, V(u,v)£j\f v . (3.38) 

n 

Here, ||(w, u)|| :— (J Ri (\Vu\ 2 + \Vv\ 2 ) dx) 1 / 2 is also a norm of D, which is equiv- 
alent to || (tt, v)\\®. Recall that (u n ,v n ) — 1 (0,0) weakly in D, by (|3.37p we also 
have J v (u n ,v n ) —> c„ and (u n ,v n ) (0,0) weakly in B. Moreover, by Lemma 
13. II we may assume that (u n ,v n ) satisfies (|3.15l) for all n£N. Then by repeat- 
ing the proof of [121 Theorem 1.3 (l)-(2)], we can prove that J' v {u n , v n ) — > as 
n — » oo. Hence Step 1 yields that (I3.20[) holds for (u n ,v n ). Combining this with 
(|3.37p . we see that (|3.20p holds for (u n) v n ). This completes the proof. □ 

Proof of Theorem 11.21 Fix any v £ (0,fi). Take a sequence (u n ,v n ) £ Af v 
such that J v (u n ,v n ) — > c v as n — > oo. Recall that E(u,v) = |Vu| 2 + |Vw| 2 

w 



rhH 2 - rh\v\ 2 , there exists R„ > such that 



E(u n ,v n ) = [ E{u n ,v n ) = \{\\u n \\ 2 Xl + IKHaJ- 

JR N \B Rn L 

Define 

( JV— 2 N-2 \ 

R n 2 Un(RnX), R n 2 V n (R n x)j, 

Then by a direct computation, we see that (u n ,v n ) £ N u and J v {u ni v n ) — > c v . 
Moreover, 

E{u„,v n )= I E(u n ,v n ) = l(\\u n \\ 2 Xl + \\v n \\l 2 ) ^ 2c„ > 0. (3.39) 

JM N \B 1 * 

By the Ekeland variational principle (see [31] Theorem 5.1] for example), there 
exists a sequence {(u n , v n )} £ M v such that 

Jis(u n ,v n ) < J u (u n ,v n ), \\(u n ,v n ) - (u n ,v n )\\ < -, (3.40) 

n 

J v (u,v) > J v (u n ,v n ) \\(u n ,v n ) - (u,v)\\, V(u, v) e 7V„. (3.41) 

n 
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Similarly as Step 2 in the proof of Lemma 13.31 we have that J u (u n ,v n ) — > c„ 
and J' v {u ni v n ) — > as n — > oo. Moreover, f|3.39[) and (|3.40[) yield that 

lim / E(u n ,v n )= lim / E(u n , v n ) = 2c u , (3.42) 



lim / E(u n ,v n ) = lim / E(u n ,v n ) = 2c„. (3.43) 

Note that (u„,i>„) are uniformly bounded in B. Then up to a subsequence, 
we assume that (u n ,v n ) {u,v) weakly in D. Then J' u (u,v) = 0. 
Case 1. (u,v) = (0,0). 

Then we can apply twice Lemma 13.31 with r = 1 and e = ±1/4 respectively, 
and there exist p + € (0, 1/4) and p~ G (—1/4, 0) such that the alternative p. 201) 
holds. By (|3.42j) - p.43[) we can rule out all possibilities other than 

f (|Vii„| 2 + |V«„| 2 )^0and f (\Vu n \ 2 + \Vv n \ 2 ) 0. (3.44) 

Now let r) € C§°(R 4 ) such that < r? < 1, r)(x) = 1 for \x\ e [3/4,5/4] and 
r)(x) = for |x| £ [1/2,3/2]. Recall that (u n ,v n ) (0,0) weakly in D, so 
u ni v n — > strongly in L 2 oc (R 4 ). Combining this with (|3.44p . we obtain that 

||(»7Un) - v*n\\ 0, ||(??fn) - v n \\ Q, as TL V oo. 
By Hardy inequality (ll.6[) . we have 

(1-*?) 2 «S , /" C*-V)vu 2 n 



K 4 l^l 2 v/r 4 |^|' 

< ||(rju n ) -u„|| 2 + 8 / u 2 n = o(l), 

Jl/2<\x\<3/2 

2 

Similarly, J" R4 = o(l). Therefore, we see from (w n ,i; n ) G M v that 
|V Wn | 2 = / i4 + 2i/ / u 2 « 2 +o(l), 

,2„,2 



V«„| 2 = / «*+2i/ / << + o(l). 

JR 4 JK 4 

From Lcmma l3.2l we may assume that | w„ | * , \v n \\ > C > 0, where C is indepen- 
dent of n. Since 1v < 1v' < 1, then it is easy to prove that there exist t n , s n > 
such that (\/Uttii, ^/s7iVn) € .M^ and (t n , s„) — > (1, 1) as n — > oo. Hence, 

c v = lim J !/ (w n ,w ri )= lim 7 (||unllAi + IKIlD 
= lim -fi„ / |Vu„| 2 + s„ / |Vu„| 2 

n—^oo 4 
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= lim L u (y/t^u n ,y/s^v n ) > m„, 

n~ > oo 

a contradiction with Lemma 13. II So Case 1 is impossible. 
Case 2. Either u = 0, u ^ or u ^ 0, v = 0. 

Without loss of generality, we assume that u ^ 0, u = 0. Note that J^,(it, 0) = 
yields 

\\u\\l 1 =\u\\<S{X 1 )-' 1 \\u\\'i^ 

which implies ||it||^ > S'(Ai) 2 . 

Case 2.1 Up to a subsequence, lim^oo \\u n — u\\ > 0. 

Denote w n = u n — u. Note that v n ) € A/^. Then by Brezis-Lieb Lemma 
(13) and Lemma [2731 we conclude that 

ll«'n||A 1 =/ w n + 2 " [ W 2 n vl + o{l), 
JR 4 JR 4 

IKIIL = / «£ + 2f / Wl 2 ^ +0 (1) 7 
JR 4 JR 4 

Similarly as above, it is easy to prove that there exist t n ,s n > such that 
(Vtn w n, y/s^Vn) G M v and (f„, s„) -> (1, 1) as n -» oo. Hence, 

d, = lim J v {u ni v n ) = lim -t(||u„||1L + IWlL) 

n — » oo n— J-oo 4 

= tII u IIa 1 + lini 7 (^ll^llA! + s nlWlA 2 ) 
> lim Jv(\fU,w n ,y/~s^v n ) > c„, 

n— ► oo 

a contradiction. So Case 2.1 is impossible. 
Case 2.2 u n u strongly in D 1 ' 2 (R N ). 

Then u 2 n ->• u 2 strongly in L 2 (E 4 ). Recall that v n ^ in £) 1,2 (M 4 ), so 
u 2 — ^ weakly in L 2 (M 4 ), which easily implies 

/ u 2 n v 2 n < f u 2 v 2 n + ( \u 2 n -u 2 \vl = o{l). 

JR 4 JR 4 JR 4 

Then we have 

IKIIa 2 = Kll + o(l) < 5(A 2 )" 2 ||« n ||l 2 + o(l), 
Since Lemma [32] yields lim n ^oo ||« n || 2 2 > 0, so lim„^oo ||w„|| 2 2 > S(X 2 ) 2 , and 

Cu = lim J v {u n ,v n )= lim + ||w„|| 2 J 

n— >oo n— yoo 4 

= \\\u\\l 1 + lim h\ Vn \\l 2 >\(s(X 1 ) 2 + S(X 2 ) 2 ), 

4 n— >oo 4 4 

a contradiction with Lemma 13.11 So Case 2.2 is impossible, and so Case 2 is 
impossible. 
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Since neither Case 1 nor Case 2 is true, we obtain that u ^ and v ^ 0. 
Since J^(u,v) — 0, so (u,v) € N v . Meanwhile, Fatou Lemma implies that 

c y < J v (u,v) < liminf J v (u n ,V n ) = c v . 

n— >oo 

so Jv{u,v) = c v . Then (|u|, \v\) G M v and J„(|u|, |u|) = c„. Since 2v < 1 
and a = /3 = 2, then by repeating the proof of Lemma |2.1[ we can prove that 
J' v {\u\, \v\) = 0. By the maximum principle, \v\ > in R 4 \ {0}. Therefore, 
(|u|, |i>|) is a positive ground state solution of (ll.20[) . 

To finish the proof, it suffices to prove c„ — > j (S'(Ai) 2 + S(A 2 ) 2 ) as v — > 0. 
From the above argument, we may assume that (u Vl v u ) is a positive ground 
state solution of (|1.20[) with c v = J v (u v ,Vy) for any v £ (0,^i). Since c v < 
j (S'(Ai) 2 + S'(A2) 2 ), we see that (u„, v v ) are uniformly bounded in D. Then 

\\u v \\\ = \u u \\ + 2v [ ulvl < StAO^IMlt + O(iz). 

JR 4 

From (|3.19|) we see that liminf,,-^ > 0' so hm inf^o II^IIai — 'S'(Ai) 2 . 

Similarly, we can prove that liminf \\v v \\\ > S'(A2) 2 , and so liminf „_>.o c„ > 
\ (S'(Ai) 2 + S{\ 2 ) 2 ) . That is, lim„^ c v = \ (5(Ai) 2 + S(X 2 ) 2 ) . This completes 
the proof. □ 



4 Proof of Theorem 11.51 : A variational pertur- 
bation approach 

In this section, we give the proof of Theorem II. 5[ and this result will be used 
in the proof of Theorem 11.31 Assume that N > 3, Ai, A2 € (0, Ajv) and (|1.5|) 
hold. Let v > 0. To obtain positive solutions of (ll.4[) . we consider the following 
modified problem 

-Au- i^ju-w 2 ,*- 1 = i/atif 1 ^, xeM. N , 
-Av- ^v-vf- 1 = vpu^v^ 1 , xeR N , (4.1) 
u(x), v(x) e D 1 ' 2 {R N ), 

where u±(x) := max{±w(a;), 0} and so does v±. The associated energy func- 
tional of (|4~T1) is 

J u{ u,v):=huf Xl + h\v\\l 2 - 1 / (u%+v%)-v[ u%vi, (4.2) 

III J M N V / J R JV 

Then it is standard to prove that J € C 1 (D,R). Define 

C^ r {R N ) := {u e Oq°(R jv ) : u is radially symmetric}, 
Dj' 2 ^) := {u G D 1 ^ 2 (R jv ) : u is radially symmetric}, 
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and U r := Dj' 2 (R N ) x Dj' 2 (R N ). Then O r is a subspace of B with norm || • || D . 
In this section, we consider the functional J„ restricted to B r . By Palais's 
Symmetric Criticality Principle, any critical points of J„ : B r — > R are radially 
symmetric solutions of ()4. 1|) . 

Without loss of generality, we assume that S(Xi) < S(\2). Then we see 
from (Ti~T2"j) that 

5(A 1 ) jV / 4 = ||zi|| Al <||^|| A2= 5(A 2 ) Ar / 4 , V M >0. (4.3) 

Define 

jA , if «eJ) 1 V)\{0}, . 

Px (u) :— < l|u|l ^i « = 1, 2. 

[O, if u = 0, 

By (fi~T7)) it is easy to prove that P Xl £ C^D 1 ' 2 ^), K). Note that P Al (u) = 1 
is equivalent to I' x ,(u)u = 0. Then by (|1.11[) - (|1.14[) it is easy to check that 

Mi :— ^S(X l ) N ^ 2 — inf I Xi (u), i = 1,2. (4.4) 

*\(«)=1 



By (fi32>(fi~T4)) we have 

/A 4 (tei) - - —\z\& = [ ^ — ) W4WI, i = 1,2. (4.5) 



' i |2 * _ / 1 1 \ II J, 1 1 2 



Note that 

7 Ai (zj) = max/ Ai (tej) = M 4) i = 1, 2, Af x < M 2 , (4.6) 
it is easily seen that there exist < to < 1 < ti such that 

I\A tz \) < M i/ 4 for t e (0,t ] U [*i,oo), i = l,2. (4.7) 

Define 

ji(t):=tz\ for <t<t u i = 1,2; 7(t, s) := OftW.TfeOO). 
Then 7(i, s) g B r for all (£, s) and there exists a constant C > such that 

max ||7(M)||d<C. (4.8) 

(M)6[0,ti]x[0,ti] 

Denote Q :— [0,ti] x [0,£i] for convenience. For v > 0, we define 
av := inf max J^fjCt, s)), d v := max J v i^j{t, s)), 

7Gr (t,s)£Q (M)GQ 



where 



| 7 GC(Q,B r ) : max || 7 (i, s)|| D < 25(A 2 ) Ar / 4 + C, 



r := 

(M)e<3 
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7 (t,s) = 7(t,s) for (t,a) £ Q\(t ,h) x (t ,ti)}. (4.9) 

The definition of a„ is different from the definitions of usual mountain-pass 
values (cf. [5]). All pathes in T are required to be uniformly bounded in D by 
2S(\2) N / 4 + C, which will play a crucial role in the proof of Lemma |4~T1 below. 

Lemma 4.1. There hold d v < do for v > and lim a v — lim d v — ao — 
d = Mi +M 2 . 

Proof. Note that z\ > in R N \ {0}, we have 

do — max Jo(7(£, s )) = max max I\ 2 (sz\). 

(t,s)e<2 te(o,ti) se(o,ti) 

Then by (|45]) - (|4~6|) we see that 

do = Jo(7(l,l))=Afi+Af 2> J o 0{t,s))<d o for (t,«) e Q\{(1,1)}. (4.10) 

Fix any v > 0. Note that there exists (t Ul s^) E Q \ {(0,0)} such that 
d v = J v (j{tis,Sv))- If {t v ,Su) — (1, 1), then we see from ()4.10j) that 

d* - X(7(l, 1)) = Jo(7(l. !)) - " / (*i) a (*i) / ' <te < ^o; 
if {t u ,s v ) 7^ (1,1), then we deduce from ()4.10|) again that 

d„ = l v (rf(tu, 8 V )) = Jo(7(tv,s v )) - v%4 I ( z i) a (4f dx < d a , 

JR N 

that is, d v < do for any v > 0. Note that 7 € T, we have a„ < g?„, that is 

limsupa^ < liminf d v < limsupd^ < c?0i a o < ^o- (4-H) 

v-yO+ i/->0+ ' v ^o+ 

On the other hand, for any "f(t,s) = (71 (t, s), 72 (t, s)) € T, we define T(7) : 
[<o,*i] x [to,tx] -> K 2 by 

T( 7 )(t, s) := (P Al (7i(*. «)) " ^a 2 (72(*, «)), ^ ( 7 i(i, «)) + Pa 2 a)) - 2) . 
By the definitions of P\ i and 7, it is easily seen that 

T(7)(i, a) = (t 2 *- 2 - s 2 *- 2 , i 2 *- 2 + s 2 *- 2 - 2) . 

Then deg(T( 7 ), [t ,h] x [t 0) ti], (0,0)) = 1. By flU} we see that for any (i,s) € 
d([t ,t 1 ]x[t ,t 1 ]),?(r/)(t,a) = T(rf){t,a) + (0,0). Therefore, deg(T( 7 ), [t , t{\ x 
[tojti]) (0,0)) is well defined and 

deg(T( 7 ),[t ,ti] x [t ,t 1 ],(0,0))=deg(T(7),[<o,ti] X Mi], (0,0)) = 1. 
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Then there exists (t2,s 2 ) G [t ,ti] 2 such that T(7)(i 2 ,S2) = (0,0), that is 
P\ 1 (li(t2, S2)) = 1 and P\ 2 (72 (*2, S2)) = 1. Combining these with (|4.4I) . we 
have 



max J (7(M)) > Jo(j(h, s 2 )) > Jo(l(h,s 2 )) 

= I\ 1 (ll( t 2;S 2 )) + /a 2 (72(*2,S 2 )) 

>M! + M2 = d . (4.12) 

Therefore, a > do- By (|4. 1 1[) one get that ao = do- 

Assume by contradiction that liminf „_>o+ a v < ^o- Then there exists e > 0, 
v n -» 0+ and j„ = ( r y n ,i, 7^,2) € T such that 

max J„„(7„(i, s)) < d - 2e. 

Recall a + (3 = 2*. By (|4.9[) and Holder inequality, there exists no large enough 
such that 



max v n 

(*,s)e<2 



(7n,i(M)+) Q (7n,2(*,s)+) /3 dx 



< C^n < s, V n > no, 



and so 



ao < max Joh n (t, s)) < max J„ n hn(t, s)) + e < d — e, V n > n , 

(t,s)GQ (t,s)£Q 

a contradiction with ao = do. Therefore, lim inf „_>o+ o, v > do. Combining this 
with (|4.11[) , we complete the proof. □ 

Recall dUnH, we define X := Z x x Z 2 C D r , and 

X 5 := {(it, v) G D r : dist((u, v), X) < S}, it := {(it, u) G D r : J u (u, v) < d}; 



S := 



5- s s(A -» w ' J 



(4.13) 



Here dist((u, u), X) := inf{||(u — w 



(<p,t/>) G X}. Define 



j'„(it,«) —supjX^w)^)^) : ftM)eB r , ||(^,0)||d = 1} • 

Lemma 4.2. Recall S in H4.13\ ). Then there exist < cr < 1 and 1/3 G (0,1), 
such that \\J v (u, v)\\ > a holds for any (it, v) G J„" n (X S \X S / 2 ) and v G (0, ^3]. 

Proof. Assume by contradiction, there exist v n — > 0+ and (u n ,v n ) G J,/"" H 
(A^Jf 5/2 ) such that ||X n K, V)ll -> 0. Then there exist [i i>n >0,i = l,2,n€ 
N such that 



< 2(5, VneN. 
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Hence (u n , v n ) are uniformly bounded in D r , and up to a subsequence, we may 
assume that (u n ,v n ) — ^ (u,v) weakly in D r . However, since (z^zz) — 1 (0,0) 
weakly in D r as fi —¥ oo, we know that X is not compact in D r . So it seems 
very difficult for us to show that u ^ and v ^ 0. To overcome this difficulty, 
let us define 



u n (x) := u n {ni >n x), v n (x) := (j, 2 £ v n (fj,2, n x). 
Note that || • || i — 1,2 are invariant with respect to the transformation tt(«) i-> 

N — 2 

fi ~ u(-) for all /i > 0. Therefore, 



< 25, 



-HIa 2 



M2, 



< 28. 



This means that (u n ,v n ) are uniformly bounded in P r . Up to a subsequence, 
we may assume that (Un,v n ) (u,u) weakly in P r ni 2 '''(K JV ") x L 2 *^). Then 
we have ||u — 2i||ai < liminf n _>oo \\u n — zl\\\ 1 < 2(5. Combining this with (|4.3|) 
and (|4.13j) . we get that u ^ 0. Similarly, u ^ 0. 

Take any e C$° r (M. N ) such that ||0|| Al = 1, we define 



cj) n {x) := fi 1 2 cj) 



fJ>l,n 



(4.14) 



"niiAi — II^IUi = 1- Since v n — > 0, by Holder inequality and Sobolev 
inequality, we easily obtain that 



lim 



v n a I (u„)+ 1 (vn)+<t> n dx 



= 0. 



Therefore, 

0= lim J (u n ,v n )(<f> n , 0) 

n— >oo 

= lim / Vu„V0„ - -rr~u n (j> n - (w„)+ ~ 1 4> n dx 

n^ooJ RN \ X \ 2 



lim / Vu n V(l) - -rr^Un^ - (u n )+ 1 4>dx 



S7uS7(f> - P^u<t) - u%~ l 4>dx, holds for any g C$° r (R N ), 



that is, —Am — yjp-u = w + 1 and u € Dj' 2 (M JV ). By testing this equation 
with m_ we see that u > 0. By the maximum principle, one has that u > in 
\ {0}, that is u is a positive solution of (ITS)) with i = 1. Then by (fI75 ]) -([TTg ]l 
we get that u G Z\. Similarly, we may prove that v £ Z 2 , that is, (u,v) G X. 
Recall that J „ n (it n , u„) < d„ n and a + f3 — 2*, we deduce from Lemma [4~T1 that 

Mi + M 2 > lim \J Vn {u n ,v n ) - — J v (u n ,v n )(u 
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■i lip (IKIIa! + IKIIiJ = T? i™ (Kill + \\vn\\l 2 



>^\Hl 1 + ^\\v\\l=M 1 + M 2 , 

which implies that all inequalities above are identities, so J Vn (u n ,v n ) — > Mi + 
M 2 and (u n ,v n ) — > (u,v) S X strongly in D r . Then 

\\(u n ,v n ) - (u,v)\\ < 5/4, for n large enough, 

and so 

\\(u n ,v n ) ~ (u n ,V n )\\ < (5/4, for n large enough, 

where 

(_ Ar ~ 2 / a; \ w-2 / £ \\ 
Ml,„ 2 ^(^— J,^2,« 2 ^ ( )) eX ' 

This contradicts with (u„, v n ) X s / 2 for any n. This completes the proof. □ 
Lemma 4.3. There exists £ (0, ^3] and e > suc/i t/iat /or any ^ € (0, 1/4], 

Jv(j(tj s )) > a u — e implies that jit, s) € X^ 2 . 

Proof. Assume by contradiction that there exist v n — > 0, e n — > and 
(^n, s n ) € Q such that 

7„„(7(*n,Sn)) >a y „ -e n and ^I i/2 , VneN. (4.15) 

Passing to a subsequence, we may assume that (t n ,s n ) — > (t,s) € Q. Then by 
Lemma 14.11 and letting n — > 00 in (|4.15j) , we have 

Ml(t,S)) > lim a Un = Mi + M 2 . 

Combining this with (|4.10[) . we obtain that (t, s) = (1, 1). Hence, 
lim ||7(t n ,s„)-7(l,l)|| D = 0. 

However, 7(1, 1) = (zj, z\) e X, which is a contradiction with (|4.15[) . □ 
Let 

£o :=mm||,^,ioa 2 |, (4.16) 

where <5, a are seen in Lemma 14.21 By Lemma |4 . 1 1 there exists £ (0, 1/4] such 
that 

\a u - d v \ < e Q , \a u — (Mi + M 2 )| < £qi Vi/£(0,4 (4.17) 
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Lemma 4.4. For any fixed v £ (0,1/2], there exists {(it n , V n )}%Li C X s n J„" 



J v {u n ,V n ) 



—¥ as n — > 00. 



Proof. Fix any 1/ <G (0, fa]. Assume by contradiction that there exists < 
l(v) < 1 such that ||J^(tt,u)|| > l(y) on X s n J,,". Then there exists a pseudo- 
gradient vector field T v in D r which is defined on a neighborhood Zj, C D r of 
X n Jj/ (cf. [21]), such that for any (u,v) e Z„, there holds 

||T„(u,u)||jj < 2min{l, || J^(u, v)\\], 
J v ( u > v )( T u(u,v)) > min{l, || J^u.tj)!!}!! J v (u,v)\\. 

Let ?7„ be a Lipschiz continuous function on ESV such that < r] v < 1, 77^ = 1 on 
X d n J J and ?7„ = on O r \Z„. Let be a Lipschiz continuous function on K 
such that < < 1, = 1 if |Z - a„| < | and £ v (l) = if |/ - a„| > e. Let 



e v (u,v) 



—T] u (u,v)^ v (J v (u,v))T u (u,v) if (u,«)6Z„, 
if (u,v)eB r \Z v , 



then there exists a global solution ^ y : D r x [0, +00) 
initial value problem 



to the following 



[Vvtw, w, 0) = (u,v). 

It is easy to see from Lemma T4.2I and (|4.16[) - (|4.17[) that %/} v has following prop- 
erties: 

(1) ip v (u, v, 9) — (u, v) if 9 = or (u, v) € B> r \Z u or | J v (u, v) — a v \ > e; 



(2) 



d<9 



< 2; 



(3) —J u (ip v (u, v,9))= l v {ip v {u,v,6)){e v {i> v {u,v,e))) < 0; 

(4) ^MMwe)) < -i{ v f nMu,v,e)ex s n(jt"\T;~ i 1 

(5) ^J„ftMu,«,0)) < -* 2 if e (xV 4 ' 2 )n (j^\j^ _£/2 ). 

Step 1. For any (i, s) g Q, we claim that there exists 9t, s G [0, +00) such that 

ip v (j(t, s),9t, s ) € J°" 6 ° , where e is seen in (|4.16[) . 

Assume by contradiction that there exists (t, s) S Q such that 

7„(^(7(i,s),e))>a I/ -e , V9>0. 

Note that eo < e, we see from Lemma 14.31 that j(t, s) 6 X s / 2 . Note that 
J„(j(t, s)) < < a w + £0, we see from the property (3) that 



a„ - e < Ju{^y{l{t, s),9)) <d u <a u + e , V6> > 0. 
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This implies ^(J v (ip v (^f(t, s), <?))) = 1. If i/> v (>y(t, s), 0) G X s for all > 0, then 

vAi'Alfa s ), <?)) = !, and llX(Vv(70, s), 0))|| > /(f) for all > 0. Then we see 
from property (4) that 

a contradiction. Thus, there exists tjS > such that i/) u (j(t, s), ttS ) ^ X s . Note 
that 7(t, s) G X" 5 / 2 , there exists < 0| >a < 2 s < 6> t . s such that i^Jj{t, s), 0} s ) G 
dX s / 2 ,Ml(t,s),9t s ) G 9X 5 and^(7(t, S ),^) G X 5 \X 5 / 2 for all 9 G (0^,0?,,). 

Then by Lemmagjjwe have HX^Ct^, s), 0))|| > <r for all £ (0} s ,0l s ). Then 
using the property (2) we have 

5/2 < \\Ml{t,s),0l s )-Ml{t,s),0l s )\\ o < 2\9l-9ll 
that is, 0\ s — 0} s > 5/4. This implies from (|4.16|) and property (5) that 

Ju (Mi(t, s), 9l s )) < JAMi(t, s), 9lj) + Jj' s —JAMu, «, 0)) do 

<a v +e - <r 2 (0t, s - 0t,s) <au+e - ^5a 2 
<a v - eo, 

which is a contradiction. 

By Step 1 we can define T(t, s) := inf{0 > : l v {^ v {^f{t, s), 9)) < a v - e } 
and let 7(t, s) :— ijj v (j(t, s), s)), then J„(j(t, s)) < — e for all (t, s) G Q. 

Step 2. We shall prove that 7 (t, s) G T. 

For any (t,s) G Q\(i ,ti) x (io,*i), by (|46]) -(|i7f ]l and (f47T6|) - (f4TTT)) . we have 

JAl(t,s)) <J m,s)) = I Xl (7i(t)) + h 2 (l2(s)) 

< Ah/4 + M 2 < Mi + M 2 - 3e < a„ - e , 

which implies that T(t, s) — and so j(t, s) — 'jit, s). From the definition of T 
in flU), it suffices to prove that ||7(f,s)|| D < 2S(X 2 ) N/i +C for all (t,s) G Q 
and T(t, s) is continuous with respect to (t, s). 

For any (t,s) G Q, if J u (j(t,s)) < a v — eo, we have T(t,s) — and so 
7 (t, s )_= 7(t, s), and by (gSJ we see that || 7 (i, s)j| D < C < 2S(A 2 ) JV/4 + C. 

If J„(7(i, s)) > a„ - eo, then 7 (i, s) G X 5 / 2 and 

a„-e Q <7 v (ip v (j(t,s),e)) <d v <a„ + £ , V0 G [0,T(t,s)), 

This implies &,(J„(^(7(*, «)> *))) = 1 for G [0, T(t, s)). H^„(7(t, »), T(t, »)) g 
X s , then there exists < 0\ s <0\ s < T(t, s) as above. Then we can prove that 
Jv^v^jit, s), 9\ s )) < a,y — eo as above, which contradicts with the definition 
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of T(t,s). Therefore, j(t,s) = ip„(j(t, s),T(t, s)) £ X s . Then there exists 
(u,v) £ X such that \\j{t,s) - {u,v)\\ <2S <C. By (|4~3l) we have 

\\j(t, a)\\n < \\(u, v)\\ D +C< 2S(\ 2 ) N/i + C. 

To prove the continuity of T(t, s), we fix any (t, s) £ Q. Assume J v {^{t, s)) < 
a„ — e first- Then T(t,s) = from the definition of T(t,s). So J^("f(t 7 s)) < 
a v — Eq. By the continuity of 7, there exists r > such that for any (t, s) £ 
(t—T, t+r) x (S— t, s+t)C]Q, there holds J v i^{t, s)) < a v —eo, that is, T(i, s) = 
for any (t, s) £ (i — r, i + r) x (s — t, s + t) C\Q, and so T is continuous at (t, s). 
Now we assume that J v ( r ^(t 1 s)) — a v — eo, then from the previous proof we see 
that 7(£, s) = ip„(7(t, s), T(t, §)) £ X s and so 

(7(f s g),T(t ) 5)))| >l(v) >0. 

Then for any u > 0, we have J v (^) v (^/\b, s),T(t, s) + ui)) < a v — Eq. By the 
continuity of i/v and 7, there exists t > such that for any (t, s) £ (t — 
r,t + t) x (s — t,s + t) C\ Q, we have J I/ (^ !/ (7(i, s), T(f, s) + w)) < a v — e , so 
^(t, s) < T(i, s) + uj. It follows that 

< limsup T(t,s) < T(t,s). 

(t, s )-j.(i,s) 



If T(i, s) = 0, we have 



lim T{t, s) = T(t, s) 

(t,s)->-(t,s) 



immediately. If T(t,s) > 0, then for any < ui < T(t,s), similarly we have 
</i/(Vv(7(A s), T(i, s) — w))) > a K — £o- By the continuity of ip v and 7 again, we 
easily obtain that 

liminf T(t,s) > T(t,s). 

(i,s)->(t,S) 

So T is continuous at (t, s). This completes the proof of Step 2. 

Now, we have proved that "f(t, s) £ T and max J u 0y(t, s)) < a v — eo, which 

(t,s)eQ 

contradicts with the definition of a„. This completes the proof. □ 

Proof of Theorem 11.51 Fix any v £ (0, 1/2]. By Lemma [4.41 there exists 
{{u n ,v n )}^ =1 C X s n J d J such that 



J v {u n ,v n ) 



as n — > 00. 



Note that there exist ^ n > 0, i = 1, 2, n £ N such that 



(u n ,v n ) - (4i,„' Z L,„) 



<2(5, VneN. 



(4.18) 



By (|4.3p . {(u„,w„),n > 1} are uniformly bounded in D r , Up to a subse- 
quence, we may assume that (u n ,v n ) — 1 (u, v) weakly in D r . As pointed out 
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before, since X is not compact in H r , it seems very difficult to prove that u ^ 
and v ^ 0. To overcome this difficulty, let us define 



N-2 N-2 

u n {x) := fj, 2 l u n (fj. 2 , n x), v n (x)--=H 2 l V n ((J,2,nX)- 



(4.19) 



Then by a direct computation, we see that || J v {u ni v n )\\ — > and || J v {u n , v % 
as n — > oo. Moreover, 



< 2(5, 



|"n - ^1 || A 



211 <2S. 



Up to a subsequence, we may assume that (u n ,v n ) — 1 (it, u) and («„,!;„) — v 
(u, t>) weakly in B r nL 2 "(R N ) x X 2 *(R N ). Then X(u,w) = and l' v {%v) = 0. 
Moreover, as in the proof of Lemma |4~2"1 we get that u ^ and v ^ 0. 
Now we claim that either v ^ or u ^ 0. 

Assume by contradiction that both v = and It = 0. Then u„ — v weakly in 
Dl' 2 (R N )nL r (R N ). Hence it is easy to prove that (u„)+ _1 ^ in L^(R N ). 
Take any 4> £ C™ r (R N ) such that ||0|| Al = 1, and <j> n is defined in QZIfy . Then 
we see from Hoder inequality and a + (3 = 2* that 



lim 

n— f oo 



< lim C 

n— >oo 



(u„)+ 1 {v n )' + (j) n dx 



(Un)+ 1 \4>\dx 



lim 

n— >oo 



R N 



0+ _1 |0|rfar 1 =0. (4.20) 



Combining this with J u (u n , v n )(<f> n , 0) — > 0, we may repeat the proof of Lemma 
and then get that u € Z\. Similarly, we can prove that v 6 Z 2 . Then 



d v > Jim. [ J v (u n ,v n ) - — y v (u n ,v n )(u n ,v n ) 



— lim (|K||^ 



V 



n\\A 2 ) 



\Vn\\l 2 



(4.21) 



a contradiction. So either j) ^ or « ^ 0. Without loss of generality, we may 
assume that v ^ 0. Note that u ^ and Jj,(u, w) = 0, then by testing (|4.1|) with 
u_ and iT_, we see from Hardy inequality (|1.6[) that u > and u > 0. By the 
maximum principle, one has that u > and v > in \ {0}. Hence, (u, z;) is 
a positive solution of ()1.4j) . which is radially symmetric. Moreover, 



> lim ( J v {u n ,v n ) - — f u (u n ,v n )(u n ,v n ) 
= T7 !™ (IKIIai + IKIlL) = ^ lip (II^Ha! + llvnllL) 
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that is, J v (u,v) < d v < dg 
proof. 



jr(S(Xx) N / 2 + S(X 2 ) N/2 ). This completes the 



□ 



5 Proof of Theorem 11.3k The case N = 3 



In this section, we give the proof of Theorem 1 1.31 Assume that N = 3, a + (3 = 
2* = 6, a > 2, P > 2, Ai,A 2 G (0,A 3 ) and v > 0. Recall from (fTTTj) that 
S(Xi) < S, we take a e € (0, 1/2) such that 



11 < S(Ai)l + S(A 2 )t < (2 - 2e )S*, 



l-eo 



(1 - 2e )5(A 2 )i > Eo 5(Ai)3, (1 - 2e )5(A!)2 > £o S(A 2 )t 
Define 

A„ := {(«,«) el : u ^ 0, v ^ 0, J^(u,«) = 0} 
as the set of nontrivial critical point of J„, and 

b v := inf J v (u, v). 



(5.1) 



(5.2) 



By Theorem ll.5l we see that for any v G (0, 1/2], AT^ 7^ 0, 6^ is well defined and 

b v <\(s{X x f + S{X 2 f) . 
Note that K v C A/"„, so b v > c„ > 0. Define 



Cq := max 



S(Ai)*+S(A 2 )« 



S(X 1 p+S(X 2 ) 

Then for any (u,v) € AC with J v {u,v) < ^(S(Xi)% + S(X 2 )%), since 
(u 6 + v 6 + 6v\u\ a \vf) = 3J u (u, v) < 5(Ai)* + S(A 2 )* 



(5.3) 



so 

K<C 0! \vf e <C . 
Then we see from Holder inequality and (|1.17l) that 

S(Xi)\u\l < \\uWl = \u\ e 6 + va f \u\ a \v\P < \u\ 6 6 + vaC \u\«, 

That is, we can obtain 

is(X 1 )\u\l<\u\t + ^aCo\u\2, 
\s(X 2 )\v\ 2 6 < \v\l + v/3C \v\l 

The following result was introduced in pQ. 



(5.4) 



(5.5) 
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Lemma 5.1. (see f3 Lemma 3.3]) Let N > 3, A, B > 0, and 6 > 2 be fixed. 
For any v > 0, let 

S u := jo- > : Aa& < a + vBo^ j . 

TTien for any e > 0, i/iere exists v\ > depending only on e,A,B,6 and N, 
such that 

N 

inf > (1 — e)A~ /or aZZ < f < 

Recall that a,/3 > 2 and 2* = 6. From Lemma f5.ll we have the following 
result trivially. 

Lemma 5.2. Recall eo in \5. then there exists v\ G (0, v 2 \ such that for any 
v G (Ojt'i) there hold 

S(\x)ai < o- + vaC Q cj^, cr>0 cr > (1 - e a )S(\x)%, (5.6) 

S(\ 2 )a* <<T + v0Co<r%, cr>0 => a > (1 - e )S{\ 2 )% , (5.7) 

Scr^ < cr + ^aC er^ , cr>Q^cr>(l- £ )S^ , (5.8) 

So-s < o- + i//3C o-^ , o- > =>> o- > (1 - e )Si. (5.9) 

The following lemma is the counterpart of Lemma T3.3I for the case N = 3, 
and the first part of the proof is similar to that of Lemma I3~3l so we do not give 
the details, but the latter part of the proof is quite different. 

Lemma 5.3. Assume that v G (0, v\). Let (u n ,v n ) € K v be a minimizing 
sequence ofb v , and (u n ,v n ) — »■ (0,0) weakly in D. Then for any r > and 
for every e G (— r, 0) U (0, r), there exists p G (£,0) U (0, e) such that, up to a 
subsequence, 

either [ (\Vu n \ 2 + \Vv n \ 2 ) ->■ or / (| Vw„| 2 + | V«„| 2 ) ->■ 0. (5.10) 

Proof. Fix any G (0,T/i). Without loss of generality, we only consider the 
case e G (0,r) (the proof for the case e G (— r, 0) is similar). Since (m„,w„) G i\ y 
is a minimizing sequence of b u , we may assume that J v {u n ,v n ) < ^(S(Xi)" s + 
S , (A 2 )i) and so (u n ,v n ) satisfy (|B~i | -([5T5 ]l for all n. Then by ([5T6 ]) - ([57f]l of 
Lemma l5.2l we have 

K|g>(l-e )S(Ai)', |v„|g> (l-eoWa) 1 , Vn G N. (5.11) 

Note that (u n ,u n ) are uniformly bounded in D and J'„(u n ,v n ) = 0. Then 
by repeating the argument of Lemma 13.31 with trivial modifications and using 
the same notations Wi >n , o~i >n , Ui^ n , Vi lTl with the same definitions as (|3.21|l - (|3.26|l . 
there exists p G (0, e) such that Ui y7l , Vi,n, i = 1,2 satisfy (|3.27|l - (|3.32|l . Moreover, 
we can prove that 

l«n|6 = l«l,n|g + |«a,n|g + 0(l), \v n f 6 = K n |« + \v 2 ,n\t + o(l). (5.12) 
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Now we claim that 

either lim (|K„|| 2 + IK„f) = or lim (||u 2 , n || 2 + \\v 2 , n \\ 2 ) = 0. (5.13) 

n— ^oo n— ^oo 

In fact, if (I5.13[) does not hold, then up to a subsequence, 
both lim (||wi,„|| 2 + ll^i.^H 2 ) > and lim (||u 2 .„|| 2 + ||i> 2 . n || 2 ) > 0. (5.14) 

n— »oo n— >oo 

We have the following several cases. 

Case 1. Up to a subsequence, both lim ||ui in || 2 > and lim ||vi.„|| 2 > 0. 

n— >oo n— >oo ' 

Recall that norms || • || — 1 , 2 are equivalent to 1 1 • 1 1 . Note that (|3.29|i - (|3.30p 
yield 



IKJm = l«i,nle + va / |«i, n rh,n|^ + °(1), (5.15) 

JR 3 

IKJi = Knll + VP I Kn|>l, n |" + 0(1). (5.16) 



Hence, both Ax := liminf \u\ „|g > and B\ := liminf \vx nli > 0- Since 

(u n ,v n ) satisfy (|5.4[) for all n, by (|5.12p and letting n — > oo in (|5 . 15[) - ()5 . 16[) . 
similarly as (|5.5p we can prove that 

S(Ai).a} < A a + vaCoAf, S(\ 2 )B? < B x + v^CqBJ , (5.17) 



Then by (j5T6l) - (|5?f|) of Lemma [5T21 we have 

A 1 >(l-e )S(\ 1 )^ B 1 >(l-e )S(X 2 )i. (5.18) 

Case 1.1. Up to a subsequence, lim || zi 2 ,n || 2 > and lim ||v 2 n|| 2 > 0. 
Then similarly as above, we can prove that 

A 2 := liminf \u 2 , n \% > (1 - e )5(Ai)t, B 2 := Uminf \v 2 , n \% > (1 -e )5(A 2 )». 

(5.19) 

Combining this with (|5.18[) . (|5.1I) and (15 . 12[) . we deduce that 

b v = lim Jr/(tt„,« n )= lim - ( + |u„|£ + <ov / |u„ I' 3 

n— s-oo n— >oo J y 

> lim i(|w„|6 + |„ n |«) > -{Ax +Bx+A 2 + B 2 ) 

n->oo J j 

>^(%)HS(A 2 )I) 

> |(s(Ai)*+£(A 2 )*) >K, 

a contradiction. So Case 1.1 is impossible. 

Case 1.2. Up to a subsequence, lim ||u 2 ,n|| 2 > and lim ||v2,n|| 2 = 0. 
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Then L 3 |M 2 , n | a |^ 2 ,n|' 9 — > as n — > oo, so (|3.3ip yields 
S(Ai)|u 2in |§<||u 2 , n ||i 1 = |u 2lW |g + (l) ) 
and so A 2 > S(Ai) 3/2 . Then we conclude from (j5T8| . (l5Tj) and (|5T2|) that 

6, > Km i(|« n |§ + |«„|g) >\{A 1 + B 1 + A 2 ) 

> (5(Ai)t + 5(A 2 )f ) + ^(Ai)t 

>|(s'(A 1 )i+5(A 2 )i) >6„ 

a contradiction. So Case 1.2 is impossible. 

Case 1.3. Up to a subsequence, lim ||u 2i „|j 2 = and lim ||v 2 n || 2 > 0. 

Then (|3~32)) yields 

S'(A 2 )|« 2 , B |2<|K n ||i a = |«2 1 „|§+o(l), 
and so B 2 > S{\ 2 ) 3/2 . Then we conclude from (|53gj| . (pTTj) and ([5T2]) that 

b u > lim r(|«n|e + Kll) >- s (A 1 +B i + B 2 ) 



>^-^(s(A 1 )I+S(A 2 )l)+is(A 

a contradiction. So Case 1.3 is impossible. 

Since none of Cases 1.1-1.3 is true, so Case 1 is impossible. 

Case 2. Up to a subsequence, lim ||ui,„|| 2 = and lim ||wi,„|| 2 > 0. 

Then similarly as Case 1.3, we have B\ > 5 , (A 2 ) 3 ' 2 . Moreover, we see from 
(|5TT1) and Q5T2J) that 

liminf |m 2 n |g = liminf |u„|g - lim |ui Jg > 0. 
Case 2.1. Up to a subsequence, lim ||u2.n|| 2 > and lim ||«2,n|| 2 > 0. 

n— »o o n— >oo ' 

Then similarly as above, we see that (|5.19|) holds, and so 
bu > lim U\u n \t + Kit) >l(Bi+A 2 + B 2 ) 



3 

2I 2 



> 1 ~ £ o 
~ 3 
1 

> 



(5(A x )i+5(A 2 )i) +^S(\ 2 )i 



±(s(\ 1 )i+S(\ 2 )i)>K, 

a contradiction. So Case 2.1 is impossible. 

Case 2.2. Up to a subsequence, lim ||u 2 ,n|| 2 > and lim ||^ 2 ,n|| 2 = 0. 
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Then similarly as Case 1.2, we have Ai > S'(Ai) 3 / 2 , and so 
&„> lim ^(Kle + Kle) > \(Bi+A 2 ) 



I(5(A 1 )l+5(A 2 )i) >6„ 



> 

- 3 

a contradiction. So Case 2.2 is impossible. 

Since neither Case 2.1 or Case 2.2 is true, so Case 2 is impossible. 
Case 3. Up to a subsequence, lim ||ui,„|j 2 > and lim ||vi,,i|| 2 = 0. 

By a similar argument as Case 2, we get a contradiction. So Case 3 is 
impossible. 

Since none of Cases 1, 2 and 3 is true, we see that (|5.14|) is impossible, that 
is, (|57l3| holds. Recall the definitions ([3^23]) - ([3^26]) of (ui, n ,v i>n ), ([510]) follows 
directly from (|5.13[) . This completes the proof. □ 

Proof of Theorem II .31 Fix any v £ (Q,l/i). Take a sequence (u n ,v n ) £ K u 
such that J u (u ni v n ) — > b v as n — > oo. Recall that E(u,v) = |Vu| 2 + |Vf| 2 — 
|u| 2 — | 2 , there exists R n > such that 

E(u n ,V n ) = [ E(u n ,V n ) = l(\\u n \\ 2 Xl + \\v n \\l 2 ). 

jR N \B Hrl L 

Define 

/ N-2 \ 

(u n (x),v n (x)) := (R n 2 u n (R n x), R n 2 v n (R n x)J , 

Then by a direct computation, we see that (it„,v„) <E K v and J„(u n ,v n ) — > b v . 
Moreover, 

/ E{u n ,v n )= [ E{u n ,v^ = \(\\u n \\l i + \\v n \\l 2 )^h u >Q. (5.20) 

Besides, we may assume that J v {umV n ) < ^(S^Ai)^ + S*^)^) and so (u n ,v n ) 
satisfy ([5~4)) - (15751 for all n. Then by ((576| - ((577)) of Lemma [5721 we have 



|u„|g>(l-eo)S(Ai)', \v n \t>(l-£ )S(A 2 )z ) VneN. (5.21) 

Note that (u n ,v n ) are uniformly bounded in D. Then up to a subsequence, 
we assume that (u ni v n ) (u,v) weakly in D. Then J' v {u n ,v n ) — implies 
Jl(u,v)=0. 

Step 1. We show that both u e^e and v ^ 0, that is (u,v) £ K v . Moreover 
J u (u, v) = b v . 

Case 1. (u,v) = (0,0). 

Then we can apply twice Lemma 15.31 with r = 1 and e = ±1/4 respectively, 
and there exist p + £ (0, 1/4) and p~ £ (—1/4, 0) such that the alternative (|5.10|) 
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holds. Then by repeating the argument of Case 1 in the proof of Theorem 11.21 

2 2 

with trivial modifications, we get that J R3 y^yy = o(l), J R3 y^jj- = o(l) and so 

S\u n \ 2 6 < f \Wu n \ 2 = f u & n + va [ \u n \ a Kf + o(l), 
jk 3 Js. 3 Jm 3 

S\Vn\l< I \Vv n \ 2 =[ V*+U0 [ \u n \ a \v n f + o(l) . 

JR3 J R 3 J R 3 

Denote A = liminfn^oo \u n \g and B = liminf, woo |v„|g, then (|5.21[) yields 
A > and _B > 0. Then by Holder inequality it is easy to prove that 

SAi < A + vaC Ai, SBi < B + vPCqB* . 

Then by (pTSll - fijl?]) of Lemma IQ1 we have 

A>{l-e )Si, B>(l-e )Si. 

So we conclude from (I5.1[) that 

K > hm \(\u n \t + \v n \t) > \(A + B)> 2 —^si 

n^co 6 6 6 

a contradiction. So Case 1 is impossible. 

Case 2. Either u = 0, v ^ or u ^ 0, v = 0. 

Without loss of generality, we assume that u ^ 0,v = 0. We see from 
Jl(u,v)(u,0) = that 

S(X,)\u\l < \\u\H = \ug 

which implies \u\ 6 6 > S'(Ai) 3 / 2 . 

Case 2.1. Up to a subsequence, lim^oo ||u„ — u\\ > 0. 

Denote w n = u n — u. Note that J^(u ni v n ) = 0. Then by Brezis-Lieb Lemma 
([7]) and Lemma l2~3l we conclude that 

IKIIa^/ w 6 n + ua[ \w n \ a \v n f + o(l), 
Jm. 3 Jm 3 

\K\\l= ! < + v&\ \w n \P\v n f+o(l). 
Jr 3 Jm. 3 

Denote C = liminf n ^oo \w n \%, then C > 0. Then by Holder inequality it is easy 
to prove that 

5(Ai)C3 < C + vaC C%, S(X 2 )B^ < B + vfiCoB^ . 
Then by (|5^6| -([5J ]) of Lemma IQ1 we have 

C>(l-e )S(X 1 )^, B>(l-s )S(X 2 )i. 
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So we conclude from (|5.1[) that 

K> lim kK\t + \vn\t) = hu\ 6 6 + lim I(|«,„|g + \v n \ 6 6 ) 

n— )-oo o o n— >oo o 

> ^(Ax)§ + i(B + C) > ^S(Xr)i + (s(Ai)* + S(A a )*) 

>^(5(A!)i+S(A 2 )i) >6„, 

a contradiction. So Case 2.1 is impossible. 
Case 2.2. u„ -> u strongly in D 1 ' 2 (E 3 ). 

Then u n u strongly in L 6 (R 3 ). Recall that u n in D 1 ' 2 (M 3 ), up to 
a subsequence, m„ — ?> u and w„ — > almost everywhere in R 3 . So Lemma [2.31 
yields 

/ \Un\ a Kf= f |ti„-u|«W + o(l)=o(l). 

Then we have 

S(\2)\v n \l<\\v n \\i 2 =K\t+0(l), 

so B > 5 , (A 2 ) 3 / 2 , and we conclude from (|5.ip that 

6, > lim + K|<>) > \(\u\% + B)>\ (S(\i)* + S(A 2 )i) > &„, 

a contradiction. So Case 2.2 is impossible, and so Case 2 is impossible. 

Since neither Case 1 nor Case 2 is true, we obtain that u ^ and v ^ 0. 
Since J' v {u,v) — 0, so € Then 

6„ < J„{u,v) = -||(u,u)||d < liminf -||(u„,u„)||d = liminf J y {u n ,v n ) = b v , 
so Jv(u,v) = and (it n ,v n ) — > (u, strongly in D. Then (|5.21|> implies that 

Mg>(l-eo)S(Ai)i, |«|§>(l-eo)5(A a )*. (5.22) 

Step 2. We show that neither u or v is sign-changing, so (\u\, \v\) is a positive 
ground state solution of (jl.4p . 

Assume by contradiction that m + ^ and u_ ^ 0. By Jl(u,v)(u±,0) = 
we obtain 

s(Ai)iu±n < imi = / 4 + ^/ M>r 

JR 3 JR3 

Then 

S(Ai)|u±H < |u±|g + i/aCo|u±|?, 
By ()5.6|) of Lemma 15.21 we have 

\u ± \ 6 6 >(l-e )S(X 1 )i. 
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So we conclude from (|5.1[) and (|5.22l) that 



bu>l(\u\t + \v\i) > 2 -^S(X 1 )i + i^5(A 2 )i 

>\(s(Xif> +5(A 2 )') >&„ 

a contradiction. So u is not sign-changing. Similarly, v is not sign-changing. 
That is, (|u|, |u|) is a solution of J„. By the maximum principle, we see that 
\u\ > and \v\ > in R 3 \ {0}. Since J v (\u\, \v\) — b u , so (|u|, \v\) is a positive 
ground state solution of (11.41) . 

Step 3. Wc show that b v -> \ {S{Xif 2 + S(X 2 ) 3/2 ) as v -> 0. 

From the above argument, we may assume that (ti„, i;„) is a positive ground 
state solution of (|1.4|) with &„ = J u (u u ,v u ) for any i/ € (0,z/i). The rest argu- 
ment is similar to that in the proof of Theorem II .21 and we omit the details. 
This completes the proof. □ 



6 Proof of Theorem 11.41 : The moving planes 
method 



In this section, we will use the moving planes method to prove Theorem II. 41 In 
the sequel, we assume that N — 3 or N — A, a + (3 — 2* , a > 2, (3 > 2 and 
Ai,A2 £ (0,Ajv). Fix any v > 0. Let (u, v) be any a positive solution of (|1.4[) . 
For A < we consider the reflection 

x = {xi,x 2 , ■ ■ ■ ,Xff) n- x x = (2X - xi,x 2 , ■ ■ ■ ,xjy), 

where x g £ A := {x g R N : x\ < A}. Define u x (x) := u(x x ) and v x (x) := v(x x ), 
then 

u(x) = u x {x), v(x) = v x (x), for x g <9£ A = {x g K w : xi = A}. 
Define w A (a;) := u A (:r) — u(je) and cr A (a;) := v A (ir) — v(x) for x g E A . Then 

w A (x) = cr A (a;) = 0, Vi g <9£ A . (6.1) 
Recall that («, w) satisfies (|1.4I) . we have 

-Aw A (a;) =-^u; A (a;) + a A (a;)w A (x) + a A (a;)cr A (izi) + Ai ( — - -L) u A (x) 
|ar| \ |x | \x\ J 

-1^2^^ + a i( x ) wX ( x ) + a x (x)(j x (x) (6.2) 
holds in E A \ {0 A }, where 

\ ( M A ) 2 *- 1 -u 2 *- 1 j^r 1 -"" 1 , s 

a A := V y + i>av l3 K ' >0, (6.3) 
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Similarly, 



a\ := vain*) 01 ' 1 ^ > 0. (6.4) 



-Aa x (x) > P^<J X {x) + b x (x)a x (x) + b x (x)w x {x) (6.5) 



holds in E A \ {0 A }, where 

u A — w u A — u 

fc 2 A := i/fifoY" 1 ( " A) ° ~ > 0. (6.7) 

Define 

n x := {x £ S A : w x (x) < 0} , A := er A (x) < 0} . 

Since u, v £ L 2 (R ) and J7 A C S A , there exists Ao < such that for any 
A < Ao, we have 

(2* -l)u r ~ 2 + va{a-l)u a - 2 V P < - ( 1 - ) S, (6.8) 

(2- - iy- + „ W - l)»V-»|| i/ , / , M) < i (l - S, (6.9) 

<-« 2 k-v-'i:.,,^, ^('-cK'-c) s! - (6 - io) 

Step 1. We claim that for any A < Ao, both w x > and er A > in S A \ {0 A }. 
Fix any A < Aq. Define w_ := max{— w x ,0} and rr A := max{— er A ,0}, then 



w 



A , er A € Z? 1,2 (M Ar ). Testing (|6.2|) with u; A and using Holder inequality and 



Hardy inequality (|1.6j) . we obtain 



/ |V W A | 2 < / ^ 2 \w X \ 2 + ( a x 



x >w x \ 2 + / a x w X a x 



+ H a 2ll L f (n x nni) \\ w -\\L2'(n$)\W-\\ L **(n>y 
When 6 > 1, we see from the mean value theorem that 
s e -t e 



s - t 



<9t°- 1 . V0<s<i. 



Recall that u A < u in i7 A and w A < u in f2 A . Since a > 2 and /3 > 2, we see 
from !|6~3 1) -(t0 1) and (|6"l) ]> -([6"7r ]) that 

Oi < (2* - l)u 2 *^ 2 + i/a(a - l)u Q -V, in ft A , 



49 



4, b% < vaPu a - 1 v f3 - 1 , inft A nft A , (6.11) 
b X < (2* - l)v r - 2 + v/3((3 - l)u a v - 2 in Sl x . 

Then we see from (|Ot and (IL~T5)| that 

From above we obtain 

S ~ ^) ^ ' VW -' 2 - Kno^)ll--ll^(^)ll--ll^(^)- (6-12) 
Similarly, testing (|6.5|) with er A we can prove that 

i (' " ^) 4 ' W -' 2 " Il^ll^(uxn^ll--II^K)H^II^(^)- (6-13) 
Let |f2| denotes the Lebesgue measure of O in M. N . If |f2 A n f2 A | > 0, then 

/ |Vw A | 2 >0, f |Vct a | 2 >0. (6.14) 
Combining this with (|6H|) - (j03l) . (fTTT3|) and f[67TO]) . we get 

< ll°all i iF (n A nQ} )ll^ll £ 4F(nfnn*) ll ' 1 ^ ,| £ !, *(nM ,l< ^ l| i a *(^) 

1 I, ^1 \ / , ^ 2 \f |Y7„„A|2 /" |Y7„A|2 



a contradiction. Hence |f2 A n f2 A \ — 0; an d so (|6.12[l - (|6.13[l yield 

/ |Vw A | 2 <0, f |W A | 2 <0. (6.15) 

This implies that |fi A | =0 and =0. That is, both w A > and cr A > in 
£ A \ {0 A }. If w A ee in S A \ {0 A }, then we see from (gTS} that 

-Aw A (i) > Ax ( — ^ - i ) u x (x) > in E A \ {0 A }, 



a contradiction. So u> A ^ in S A \ {0 A }. Then by the maximum principle, we 
conclude that w x > in S A \ {0 A }. Similarly, cr A > in S A \ {0 A }. 

Step 2. Define A* = sup {A < : w x > 0, o x > in E A \ {0 A }, V A < A}. Then 
we claim that A* = 0. 
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Assume by contradiction that A* < 0. Clearly we have both w x > and 
a x > in £ A \ {0 A }. By a similar argument as in Step 1, in fact we have 
both w x " > and a x " > in £ A * \ {0 A *}. Take e > such that 



1 



1 



Ai 



A 



/v 



1 



A, 



£ < Z m in il 1 - Zii. 5, 7 1 - ^ 5, -^ l-^ l-^ S 



A 



TV 



1 



16 



Ai 



A 



TV 



A 2 



A 



N 



Then there exists small Si € (0, |A*|) such that for any A e [A*, A* + Si), there 
hold 



(2* - l)u 2 '- 2 + va{a - l)u Q -V 
(2* - l)v 2 '- 2 + v8(/3 - l)u a v?- 2 

< e 



L«/2(£*\£ A *) 



< e, 



< e, 



(6.16) 
(6.17) 
(6.18) 



(va,S\ 2 \\u a - x ^- x \\ 

Meanwhile, since w x > and <r x > in E A \ {0 A }, by convergence almost 
everywhere and thereby in the measure sense of (w x ,v x ) — > (w x ,v x ) in S A , 
we have that 



lim 

A— >A* 

lim 

A->A* 



(2* - l)u r - 2 + va{a - 1K"V 
(2* - l)v 2 '- 2 + vB{(3 - l)u a v p - 2 



L«/ 2 (^n£ A *) 
= 0. 



= 0, 



= 0, 



Then there exists 82 € (0, Si) such that for any A G [A*, A* + S2), there hold 

<£, 



(2* - l)u r - 2 + va(a - l)u Q -V 
(2* - l)v 2 '- 2 + 1/0(0 - l)u t V 3-2 



L«/ 2 (n*n£ x *) 



(z/a/3) 



2 IL,"-i„/3-i| 



L«/ 2 (n£n£ x *) 
„ < e. 



<£, 



lL JV / 2 (n A nn A ns i 

Recall that f2 A C S A . Combining these with (I6.16l) - (|6.18p . we see that 
(|6.10[) hold for any A E [A*, A* + S%). Then repeating the proof of Step 1, we 
conclude that for any A € [A*, A* + S 2 ), w x > and a x > in E A \ {0 A }, which 
contradicts with the definition of A*. Therefore A* = 0. 



Step 3. We claim that both u and v are radially symmetric with respect to the 
origin. 

With the help of Steps 1 and 2, this argument is standard. Since A* = 0, then 
we can carry out the above procedure in the opposite direction, namely moving 
the parallel planes in the negative Xi direction from positive infinity. Then they 
must stop at the origin again, and so we get the symmetry of both u and v with 
respect to in the xi direction by combining the two inequalities obtained in 
the two opposite directions. Since the direction can be chosen arbitrarily, we 
conclude that both u and v are radially symmetric with respect to the origin. 
This completes the proof of Theorem 11.41 □ 
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7 Uniqueness results for the special case Ai = A2 

and a = (3 = y 

When Xi — A 2 , some uniqueness results about ground state solutions of (|1.27|) 
were obtained by the authors in [T^l [13]. We remark that, by using the same 
ideas as in [H1[T3], these results also hold for problem (|1.4I) if we assume N > 4, 
Ai = A2 and a — (i — \. First we consider the case N = 4, then we have the 
following result, which improves Theorem 11.21 in case Ai = A2. 

Theorem 7.1. Assume that N = 4, Ai = A 2 € (0, 1), a = (5 = 2 and v > 0. 

(l)Ifu^ 1/2, then for any (J, > 0, ((1 + 2^)- 1 / 2 z i j (1 + 2v)~ x l' 1 z]^ is a 
ground state solution of Ji.^p , wit/i 

C = J„ ((1 + ^r 1/2 zl (1 + 2^)- 1 / 2 z^) = _l_5(Ax) 2 . (7.1) 

Moreover, the set {((1 + 2zy)" 1 / 2 2 / l I , (1 + 2^)" 1 / 2 z / 1 I ) : zi > 0} contains all 
positive ground state solutions of 

fSJ If v = 1/2, i/iera /or any /x > and 9 G (0,7r/2) } (sin 02* cos 6 z 1 ) is 
a ground state solution of jl-4\ ) and c\/2 — ^(Ai) 2 . Moreover, the set 
{(sin^z*, co s 9 z z) : /i > 0, 9 G (0,7r/2)} contains all positive ground state 
solutions of dl-4\ )- 

Proof. (1) This result can be obtained by repeating the proofs of [12l Theorem 
1.1 and Theorem 1.2] with trivial modifications. We omit the details. 

(2) This result can be obtained by repeating the proofs of Theorem 13. 11 (2) 
with trivial modifications. We omit the details. □ 

Remark 7.1. As pointed out in the Introduction, by we know that Zi 
contains all positive solutions of h 1.8\) . Here, for the case where N — 4, Ai = 
A2 G (0, 1), a = /3 = 2, v > and v ^ 1/2, we conjecture that the set {((1 + 
2v)~ 1 / 2 zj l , (1 + 2u)~ 1 / 2 z 1 ^) : /i > 0} contains all positive solutions of JJ.^P - 

Now we consider the case N > 5. Denote P—\ f° r simplicity. Consider 
' V -1 + pukt-H* = 1, 

< pvkili- 1 + = 1, (7.2) 
k > 0, I > 0. 

Let v > 0. By a direct computation, it was proved in [131 Lemma 2.1] that 
there exists (ko, Iq), such that 

(k ,l ) satisfies (|7.2[) and k = minjfc : (k, I) is a solution of (I7.2[) }. (7.3) 

Then we have the following uniqueness result. 
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Theorem 7.2. Assume that N > 5, Ai = A 2 G (0,Ajy) and a = /3=p=^. 7/ 
> j|>, i/ien /or any fj, > 0, (v^o^i, Vh) z jj,) * s a positive ground state solution of 
J-M| ). Moreover, the set {(s/koz^, : fi > 0} contains all positive ground 

state solutions of |7^| ). 

Proof. This result can be obtained by repeating the proofs of [T31 Theorem 1.1 
and Theorem 1.2] with trivial modifications. We omit the details. □ 
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